cross-component dynamic range adjustment system and method (cc-dra) in video encoding

专利摘要:
A method of processing video data including receiving video data, determining luma scaling parameters for each of a plurality of code word value ranges for luminance components of the video data, performing a dynamic range adjustment process on the luminance components using the luma scale parameters, determine chroma scale parameters for chrominance components of the video data using a function of the luma scale parameters, and perform a dynamic range adjustment process on the chrominance components of the video data using chroma scale parameters.
公开号:BR112020003037A2
申请号:R112020003037-6
申请日:2018-08-21
公开日:2020-08-04
发明作者:Dmytro Rusanovskyy；Adarsh Krishnan Ramasubramonian；Marta Karczewicz
申请人:Qualcomm Incorporated；
IPC主号:

专利说明:

[0001] [0001] This application claims the benefit of US Provisional Application No. 62 / 548,236, filed on August 21, 2017, and US Application Number 16 / ___, ___ filed on August 20, 2018, the total content of which is incorporated by reference here. TECHNICAL FIELD
[0002] [0002] This disclosure refers to video processing. FUNDAMENTALS
[0003] [0003] Digital video resources can be incorporated into a wide range of devices, including digital television, digital direct transmission systems, wireless transmission systems, personal digital assistants (PDAs), laptops or desktop computers, tablet computers , e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, satellite or cellular radio phones, so-called “smart phones”, video teleconferencing devices, devices video transmission and the like. Digital video devices implement video encoding techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, part 10, Advanced encoding (AVC), ITU-T H.265, high-efficiency video encoding (HEVC) and extensions of such standards. Video devices can transmit, receive, encode, decode and / or store digital video information more efficiently, implementing these video encoding techniques.
[0004] [0004] Video encoding techniques include spatial prediction (intra-image) and / or temporal prediction (inter-image) to reduce or remove the redundancy inherent in video sequences. For block-based video encoding, a video slice (for example, a video frame or part of a video frame) can be partitioned into video blocks, which can also be called tree blocks, units of video. encoding (CUs) and / or encoding nodes. The video blocks in an intra-coded slice (I) of an image are encoded using spatial prediction in relation to reference samples in neighboring blocks in the same image. The video blocks in a slice of an encoded image (P or B) can use spatial prediction in relation to reference samples in neighboring blocks in the same image or temporal prediction in relation to reference samples in other reference images. The images can be called frames, and the reference images can be called frames of reference.
[0005] [0005] The spatial or temporal prediction results in a predictive block for a block to be coded. Residual data represents pixel differences between the original block to be encoded and the predictive block. An inter-coded block is coded according to a motion vector that points to a block of reference samples forming the predictive block and the residual data indicating the difference between the coded block and the predictive block. An intra-coded block is coded according to an intra-coded mode and the residual data. For additional compression, residual data can be transformed from the pixel domain to a transform domain, resulting in residual transform coefficients, which can be quantized. The quantized transform coefficients, initially arranged in a two-dimensional matrix, can be scanned to produce a one-dimensional vector of transform coefficients and entropy coding can be applied to obtain even more compaction.
[0006] [0006] The total number of color values that can be captured, encoded and displayed can be defined by a range of colors. A color range refers to the color range that a device can capture (for example, a camera) or reproduce (for example, a viewfinder). Generally, color ranges differ from device to device. For video encoding, a predefined color gamut for video data can be used so that each device in the video encoding process can be configured to process pixel values in the same color gamut. Some color ranges are defined with a wider range of colors than the color ranges that are traditionally used for video encoding. Those color ranges with a wider range of colors can be called the wide range of colors (WCG).
[0007] [0007] Another aspect of video data is the dynamic range. The dynamic range is usually defined as the ratio of maximum to minimum brightness (for example,
[0008] [0008] This disclosure is directed to the field of video signal encoding with representations of high dynamic range (HDR) and wide color range (WCG). More specifically, in some examples, this disclosure describes the signaling and operations applied to video data in certain color spaces to allow for more efficient compression of HDR and WCG video data. For example, according to some examples, the compression efficiency of hybrid-based video encoding systems used to encode HDR and WCG video data can be improved.
[0009] [0009] This disclosure describes examples of techniques and devices for performing cross-component dynamic range adjustment of the video data chrominance components. In one example, this disclosure describes the derivation of a scale parameter for adjusting the dynamic range of luma components of video data. In one example, one or more scale parameters for adjusting the dynamic range of chroma components can be derived as a function of the luma scale parameters. By using the luma scale factors to derive scale parameters for chroma components, the amount of visual distortion in the decoded video data can be reduced. In other examples, scale factors for chroma components can be derived using a function of one or more luma scale factors, chroma quantization parameter (QP) values and / or color container parameters (for example, defined transfer functions for colored containers).
[0010] [0010] The techniques described in this document can be used in conjunction with video codecs that operate according to a video encoding standard. Sample video encoding standards may include H.264 / AVC (Advanced Video Encoding), H.265 / HEVC (High Efficiency Video Encoding), H.266 / VVC (Versatile Video Encoding), and others standards, which are configured to encode and decode HDR and / or WCG content.
[0011] [0011] In an example of the disclosure, a method of processing video data comprises receiving video data, determining luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data , perform a dynamic range adjustment process on the luminance components using the luma scale parameters, determine chroma scale parameters for chrominance components of the video data using a function of the luma scale parameters, and perform a range adjustment process dynamics in the chrominance components of video data using chroma scale parameters.
[0012] [0012] In another example of the disclosure, a device configured to process video data comprises a memory configured to store the video data, and one or more processors in communication with the memory, the one or more processors configured to receive the data video, determine luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data, perform a dynamic range adjustment process on the luminance components using the luma scale parameters, determine chroma scale parameters for chrominance components of video data using a function of the luma scale parameters, and perform a dynamic range adjustment process on the chrominance components of video data using chroma scale parameters.
[0013] [0013] In another example of the disclosure, an apparatus configured to process video data comprises means for receiving video data, means for determining luma scale parameters for each of a plurality of code word value ranges for components of luminance of video data, means for performing a dynamic range adjustment process on luminance components using luma scale parameters, means for determining chroma scale parameters for chrominance components of video data using a function of luma scale parameters , and means to perform a dynamic range adjustment process on the chrominance components of the video data using chroma scale parameters.
[0014] [0014] In another example, this disclosure describes a non-transitory, computer-readable storage medium that stores instructions that, when executed, cause one or more processors on a device configured to process video data to receive video data, determine luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data, perform a dynamic range adjustment process on the luminance components using the luma scale parameters, determine chroma scale for chrominance components of video data using a function of the luma scale parameters, and perform a dynamic range adjustment process on the chrominance components of video data using chroma scale parameters.
[0015] [0015] Details of one or more examples are presented in the attached drawings and the description below. Other features, objects and advantages will be evident from the description, drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] [0016] Figure 1 is a block diagram that illustrates an example of a video encoding and decoding system configured to implement the dissemination techniques.
[0017] [0017] Figures 2A and 2B are conceptual diagrams that illustrate an example of a binary tree structure (QTBT), and a corresponding coding tree unit (CTU).
[0018] [0018] Figure 3 is a conceptual drawing that illustrates the concepts of data 1 - 1DR.
[0019] [0019] Figure 4 is a conceptual diagram that illustrates an example of color ranges.
[0020] [0020] Figure 5 is a flow diagram that illustrates an example of 1 - 1DR / WCG representation conversion.
[0021] [0021] Figure 6 is a flow diagram that illustrates an example of HDR / WCG reverse conversion.
[0022] [0022] Figure 7 is a conceptual diagram illustrating an example of Electro-optical Transfer Functions (EOTF) used for converting video data (including SDR and HDR) from perceptually uniform code levels to linear luminance.
[0023] [0023] Figure 8 is a conceptual diagram that illustrates an example of an output curve for an EOTF.
[0024] [0024] Figures 9A and 9B are conceptual diagrams that illustrate a visualization of color distribution in two examples of color ranges.
[0025] [0025] Figure 10 is a batch of a luma-controlled chroma scale function (LCS).
[0026] [0026] Figure 11 is a block diagram illustrating an example of an HDR / WCG conversion apparatus that operates according to the techniques of this disclosure.
[0027] [0027] Figure 12 is a block diagram that illustrates an example of an HDR / WCG reverse conversion apparatus according to the techniques of this disclosure.
[0028] [0028] Figure 13 shows an example of a dynamic range adjustment mapping (DRA) function.
[0029] [0029] Figure 14 shows an example of linearization of DRA scale parameters.
[0030] [0030] Figure 15 shows an example of a set of sigmoid functions.
[0031] [0031] Figure 16 shows an example of smoothing DRA scale parameters.
[0032] [0032] Figure 17 is a block diagram that illustrates an example of a video encoder that can implement techniques of this disclosure.
[0033] [0033] Figure 18 is a block diagram that illustrates an example of a video decoder that can implement techniques of this disclosure.
[0034] [0034] Figure 19 is a flow chart showing an example of the video processing technique of the disclosure.
[0035] [0035] Figure 20 is a flowchart showing another example of the video processing technique of the disclosure. DETAILED DESCRIPTION
[0036] [0036] This disclosure is related to the processing and / or encoding of video data with representations of high dynamic range (HDR) and wide color range (WCG). In one example, the techniques of this disclosure include techniques for determining dynamic range adjustment parameters for chroma components of video data components as a function of dynamic range adjustment parameters for luma components. The techniques and devices described here can improve compression efficiency and reduce distortion in hybrid-based video encoding systems used to encode video data, including HDR and WCG video data.
[0037] [0037] Figure 1 is a block diagram illustrating an example of a video encoding and decoding system 10 that can use techniques of this disclosure. As shown in Figure 1, system 10 includes a source device 12 that provides encoded video data to be decoded at a later time by a destination device 14. In particular, source device 12 provides video data for the target device 14 via a computer-readable medium 16. The source device 12 and the target device 14 can comprise any of a wide range of devices, including desktop computers, notebook computers (i.e. laptops), type computers tablets, decoders, telephone devices, such as so-called “smart” phones, so-called “smart” pads, television, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiving device , or the like. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication.
[0038] [0038] The target device 14 can receive the encoded video data to be decoded via the computer readable medium 16. The computer readable medium 16 can comprise any type of medium or device capable of moving the encoded video data from the device source 12 to the target device 14. In one example, the computer-readable medium 16 may comprise a communication medium to allow the source device 12 to transmit encoded video data directly to the target device 14 in real time. The encoded video data can be modulated according to a communication standard, such as a wireless or wired communication protocol, and transmitted to the destination device 14. The communication medium can comprise any wireless or wired communication medium. wire, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations or any other equipment that may be useful to facilitate communication from the source device 12 to the destination device
[0039] [0039] In other examples, the computer-readable medium 16 may include non-transitory storage media, such as hard drive, flash drive, compact disc, digital video disc, Blu-ray disc or other computer-readable media. In some examples, a network server (not shown) can receive encoded video data from the source device 12 and provide the encoded video data to the destination device 14, for example, via network transmission. Likewise, a computing device in a medium production facility, such as a disk record facility, can receive encoded video data from the source device 12 and produce a disc containing the encoded video data. Therefore, the computer-readable medium 16 can be understood as including one or more computer-readable medium in various ways, in various examples.
[0040] [0040] In some examples, encrypted data can be sent from the output interface 22 to a storage device. Likewise, encrypted data can be accessed from the storage device via the input interface. The storage device can include any of a variety of data storage media distributed or accessed locally such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory or any other digital storage suitable media to store encoded video data. In an additional example, the storage device can correspond to a file server or other intermediate storage device that can store the encoded video generated by the source device 12. The destination device 14 can access stored video data from the storage device streaming or downloading. The file server can be any type of server capable of storing encoded video data and transmitting encoded video data to the target device 14. The sample file servers include a web server (for example, for a website), an FTP server, network-attached storage (NAS) or a local disk drive. The target device 14 can access the encoded video data through any standard data connection, including an Internet connection. This can include a wireless channel (for example, a Wi-Fi connection), a wired connection (for example, DSL, cable modem, etc.) or a combination of both that is suitable for accessing encoded video data stored on a file server. The transmission of encoded video data from the storage device may be a streaming transmission, a downloadable transmission or a combination thereof.
[0041] [0041] The techniques of this disclosure are not necessarily limited to wireless applications or configurations. The techniques can be applied to video encoding in support of any of several multimedia applications, such as broadcast television broadcasts, cable television broadcasts, satellite television broadcasts, video streams over the Internet, as dynamic adaptive streaming over HTTP (DASH), digital video encoded on a data storage medium, digital video decoding stored on a data storage medium or other applications. In some examples, system 10 can be configured to support unidirectional or bidirectional video transmission to support applications such as video transmission, video playback, video transmission and / or video telephony.
[0042] [0042] In the example in Figure 1, source device 12 includes video source 18, video encoding unit 21, which includes video preprocessor unit 19 and video encoder 20, and output interface 22. The destination device 14 includes input interface 28, video decoding unit 29, which includes video post-processor unit 31 and video decoder 30, and display device
[0043] [0043] The illustrated system 10 of Figure 1 is just an example. Techniques for processing HDR and WCG video data can be performed by any video encoding and / or digital video decoding device. In addition, the techniques of this disclosure can also be performed by a video pre-processor and / or video post-processor (for example, video pre-processor unit 19 and video post-processor unit 31). In general, a video preprocessor can be any device configured to process video data before encoding (for example, before HEVC encoding). In general, a video post processor can be any device configured to process video data after decoding (for example, after HEVC decoding). The source device 12 and destination device 14 are just examples of such encoding devices where the source device 12 generates encoded video data for transmission to the destination device 14. In some examples, devices 12, 14 may operate in a substantially symmetrical way so that each of the devices 12, 14 includes video encoding and decoding components, as well as a video pre-processor and a video post-processor (e.g., video pre-processor unit 19 and video post-processor unit 31, respectively). Consequently, system 10 can support unidirectional or bidirectional video transmission between video devices 12, 14, for example, for video streaming, video playback, video transmission or video telephony.
[0044] [0044] The video source 18 of the source device 12 may include a video capture device, such as a video camera, a video file containing previously captured video and / or a video feed interface for receiving video from a video content provider. As an additional alternative, video source 18 can generate data based on computer graphics such as the source video or a combination of live video, archived video and computer generated video. In some cases, if the video source 18 for a video camera, the source device 12 and the destination device 14 can form so-called camera phones or video phones. As mentioned above, however, the techniques described in this disclosure may apply to video encoding and processing in general, and can be applied to wireless and / or wired applications. In each case, the captured, pre-captured or computer generated video can be encoded by the video encoding unit 21. The encoded video information can then be sent via the output interface 22 to a computer-readable medium 16.
[0045] [0045] Input interface 28 of destination device 14 receives computer readable medium information 16. Computer readable medium information 16 may include syntax information defined by video encoder 20, which is also used by the decoding unit video 29, which includes elements of syntax that describe characteristics and / or processing of blocks and other encoded units, for example, groups of images (GOPs). The display device 32 displays the decoded video data for a user and can comprise any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma monitor , an organic light-emitting diode (OLED) or other type of display device.
[0046] [0046] As illustrated, the video preprocessor unit 19 receives video data from video source 18. The video preprocessor unit 19 can be configured to process video data to convert it into a suitable format for encoding with video encoder 20 For example, the video preprocessor unit 19 can perform dynamic range compression (for example, using a non-linear transfer function), converting colors into a more color space compact or robust, dynamic range adjustment and / or conversion from integer representation to fluctuation. The video encoder 20 can perform video encoding on the video data emitted by the video preprocessor unit 19. The video decoder 30 can perform the reverse of video encoder 20 to decode the video data, and the unit video post processor 31 can perform the reverse operation of the video preprocessor unit 19 to convert the video data into a format suitable for display. For example, the video post-processor unit 31 can perform full-number to floating conversion, color conversion from the compact or robust color space, reverse adjustment of the dynamic range and / or reverse compression of the dynamic range to generate data from suitable for display.
[0047] [0047] Video encoder 20 and video decoder 30 can be implemented as any of a variety of suitable encoder circuitry sets, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable port arrays (FPGAs), discrete logic, software, hardware, firmware or any combination thereof. When the techniques are partially implemented in the software, a device can store instructions for the software in a non-transitory and suitable computer-readable medium and execute the instructions on the hardware using one or more processors to perform the techniques in this disclosure. Each of the video encoders 20 and video decoders 30 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.
[0048] [0048] The video preprocessor unit 19 and video post processor unit 31 each can be implemented as a variety of suitable encoder circuitry, such as one or more microprocessors, DSPs, ASICs, FPGAs, logic discrete, software, hardware, firmware or any combination thereof. When the techniques are partially implemented in the software, a device can store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions on the hardware using one or more processors to perform the techniques in this disclosure. As discussed above, video preprocessor unit 19 and video postprocessor unit 31 are separate devices from video encoder 20 and video decoder 30, respectively. In other examples, the video preprocessor unit 19 can integrate with the video encoder 20 in a single device and the video post processor unit 31 can be integrated with the video decoder 30 in a single device.
[0049] [0049] In some examples, video encoder 20 and video decoder 30 operate according to a video compression standard, such as ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) and / or ITU-T H.265, also known as High Efficiency Video Coding (HEVC) or extensions to it. In other examples, video encoder 20 and video decoder 30 can operate in accordance with other proprietary or industry standards, such as the Joint Exploration Test Model (JEM). The techniques of this disclosure, however, are not limited to any specific coding standard.
[0050] [0050] The HEVC standard was finalized by the joint video coding collaboration team (JCT-VC) of the ITU-T video coding expert group (VCEG) and ISO / IEC image expert group (MPEG). A draft HEVC work specification, and referred to below as HEVC WD, is available at http: //phenix.int- evry.fr/jct/doc_end_user/documents/15_Geneva/wg11/JCTVC- 01003-v2.zip.
[0051] [0051] Recently, a new video encoding standard, known as the Versatile Video Encoding (VVC) standard, is being developed by the Joint Video Specialists (JVET) Team of VCEG and MPEG. An initial draft of the VVC is available in document JVET-J1001 “Versatile video encoding (Outline 1)” and its description of the algorithm is available in document JVET-J1002 “Description of the algorithm for versatile video encoding and test model 1 (VTM 1)".
[0052] [0052] In HEVC and other video encoding standards, a video sequence usually includes a series of images. Photos can also be called "frames". An image can include three sample matrices, called SL, S cb and Scr. SL is a two-dimensional matrix (ie, a block) of luma samples. Scb is a two-dimensional matrix of Cb chrominance samples. Scr is a two-dimensional matrix of Cr chrominance samples. Chrominance samples can also be referred to here as “chroma” samples. In other cases, an image may be monochromatic and may include only one matrix of luma samples.
[0053] [0053] C = Video encoder 20 can generate a set of encoding tree units (CTUs). Each of the CTUs can comprise a luma sample code tree block, two corresponding chroma sample code tree blocks, and syntax structures used to code the samples of the coding tree blocks. In a monochrome image or an image that has three separate color planes, a CTU can comprise a single coding tree block and syntax structures used to encode the samples in the coding tree block. A coding tree block can be an N x N block of samples. A CTU can also be referred to as a “tree block” or a “larger coding unit” (LCU). HEVC CTUs can be broadly analogous to macroblocks from other video encoding standards, such as H.264 / AVC. However, a CTU is not necessarily limited to a specific size and can include one or more encoding units (CUs). A slice can include an integer number of CTUs requested consecutively in the crawl scan.
[0054] [0054] This disclosure may use the term "video unit" or "video block" to refer to one or more sample blocks and structures and syntax used to encode samples from the one or more sample blocks. Examples of types of video units Examples may include CTUs, CUs, PUs, transform units (TUs) in HEVC, or macroblocks, macroblock partitions, and so on in other video encoding standards.
[0055] [0055] To generate an encoded CTU, the video encoder 20 can recursively perform quadratic tree partitioning in the encoding tree blocks of a CTU to divide the encoding tree blocks into encoding blocks, hence the name “tree units encoding ”. A coding block is an N x N block of samples. A CU can comprise a luma sample coding block and two corresponding chroma sample coding blocks of an image that has a luma sample matrix, a Cb sample matrix and a Cbr sample matrix, and structures and syntax used to encode samples of the coding blocks. In a monochrome image or an image that has three separate color planes, a CU can comprise a single coding block and structures and syntax used to encode the samples in the coding block.
[0056] [0056] Video encoder 20 can partition an encoding block of a CU into one or more prediction blocks. A prediction block can be a rectangular sample block (that is, square or non-square) to which the same prediction is applied. A CU prediction unit (PU) can comprise a luma sample prediction block, two corresponding chroma sample prediction blocks from an image, and structures and syntax used to predict the prediction block samples. In a monochrome image or an image that has three separate color planes, a PU can comprise a single prediction block and structures and syntax used to predict the samples in the prediction block. The video encoder 20 can generate predictive luma, Cb and Cr blocks for luma, Cb and Cr prediction blocks of each CU of the CU.
[0057] [0057] As another example, video encoder 20 and video decoder 30 can be configured to operate according to JEM / VVC. According to JEM / VVC, a video encoder (such as video encoder 20) partitions an image into a plurality of encoding tree units (CTUs). The video encoder 20 can partition a CTU according to a tree structure, such as a binary tree structure (QTBT). The JT QTBT framework removes the concepts of various partition types, such as the separation between CUs, PUs, and HEVC TUs. A JEM QTBT structure includes two levels: a first level partitioned according to partitioning of the quaternary tree, and a second level partitioned according to partitioning of the binary tree. A root node of the QTBT structure corresponds to a CTU. Leaf nodes of the binary trees correspond to the encoding units (CUs).
[0058] [0058] In some examples, video encoder 20 and video decoder 30 may use a single QTBT structure to represent each of the luminance and chrominance components, while in other examples, video encoder 20 and video decoder 30 can use two or more QTBT structures, such as a QTBT structure for the luminance component and another QTBT structure for both chrominance components (or two QTBT structures for the respective chrominance components).
[0059] [0059] Video encoder 20 and video decoder 30 can be configured to use quaternary tree partitioning by HEVC partitioning, QTBT according to JEM / VVC, or other partitioning structures. For explanatory purposes, the description of the techniques of this disclosure is presented in relation to QTBT partitioning. However, it should be understood that the techniques of this disclosure can also be applied to video encoders configured to use quaternary tree partitioning, or other types of partitioning as well.
[0060] [0060] Figures 2A and 2B are conceptual diagrams that illustrate an example of a binary tree structure (QTBT) 130, and a corresponding coding tree unit (CTU)
[0061] [0061] In general, CTU 132 of Figure 2B can be associated with parameters that define corresponding block sizes for QTBT 130 structure nodes in the first and second levels. These parameters can include a CTU size (representing a CTU size 132 in samples), a minimum quaternary tree size (MinQTSize, representing a minimum allowed quaternary tree leaf node size), a maximum binary tree size (MaxBTSize, representing a maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, representing a maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, representing the tree leaf node size minimum allowed binary).
[0062] [0062] The root node of a QTBT structure corresponding to a CTU can have four child nodes at the first level of the QTBT structure, each of which can be partitioned according to the quaternary tree partitioning. That is, the first level nodes are leaf nodes (without child nodes) or have four child nodes. The QTBT 130 structure example represents nodes, including the parent node and the child nodes with solid lines for branches. If the nodes of the first level are not larger than the maximum allowed size of the root node of the binary tree (MaxBTSize), they can be further partitioned by the respective binary trees. The division of the binary tree of a node can be iterated until the nodes resulting from the division reach the minimum allowed node size of the binary tree leaf (MinBTSize) or the maximum allowed depth of the binary tree (MaxBTDepth). The QTBT 130 structure example represents nodes as having dashed lines for branches. The leaf node of the binary tree is referred to as a coding unit (CU), which is used for prediction (for example, intra-image or inter-image prediction) and transformation, without any further partitioning. As discussed above, UCs can also be called "video blocks" or "blocks".
[0063] [0063] In an example of QTBT structure partitioning, the CTU size is set to 128 x 128 (luma samples and two corresponding 64 x 64 chroma samples), MinQTSize is set to 16 x 16, MaxBTSize is set to 64 x 64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. Quaternary tree partitioning is applied to the first CTU to generate quaternary tree leaf nodes. Quaternary tree leaf nodes can have a size of 16 x 16 (that is, MinQTSize) to 128 x 128 (that is, the CTU size). If the leaf node of the quaternary tree is 128 x 128, it will no longer be divided by the binary tree, since the size exceeds MaxBTSize (that is, 64 x 64, in this example). Otherwise, the leaf node of the quaternary tree will still be partitioned by the binary tree. Therefore, the leaf node of the quaternary tree is also the root node for the binary tree and has the depth of the binary tree as 0. When the depth of the binary tree reaches MaxBTDepth (4, in this example), no further division is allowed. When the binary tree node has a width equal to MinBTSize (4, in this example), it implies that no additional horizontal splitting is allowed. Likewise, a binary tree node having a height equal to MinBTSize implies that no additional vertical splitting is allowed for that binary tree node. As noted above, the leaf nodes of the binary tree are referred to as CUs, and are further processed according to prediction and transformed without further partitioning.
[0064] [0064] The video encoder 20 can use intra-prediction or inter-prediction to generate the predictive blocks for a PU. If the video encoder 20 uses intra-prediction to generate the predictive blocks of a PU, the video encoder 20 can generate the predictive blocks of the PU based on decoded samples of the image associated with the PU.
[0065] [0065] If the video encoder 20 uses inter-prediction to generate the predictive blocks of a PU, the video encoder 20 can generate the predictive blocks of the PU based on decoded samples of one or more images except the image associated with the PU. Inter-prediction can be unidirectional inter-prediction (ie, uniprediction) or bidirectional inter-prediction (ie, biprediction). To perform uniprediction or biprediction, video encoder 20 can generate a first reference image list (RefPicList0) and a second reference image list (RefPicList1) for a current slice.
[0066] [0066] Each of the reference image lists can include one or more reference images. When using uniprediction, video encoder 20 can search for reference images in or both RefPicList0 and RefPicList1 to determine a reference location within a reference image. In addition, when using uniprediction, the video encoder 20 can generate, based at least in part on samples corresponding to the reference site, the sample blocks predictive for the PU. In addition, when using uniprediction, video encoder 20 can generate a single motion vector that indicates a spatial shift between a PU prediction block and the reference site. To indicate the spatial displacement between a PU prediction block and the reference location, a motion vector can include a horizontal component that specifies a horizontal displacement between the PU prediction block and the reference location and can include a vertical component which specifies a vertical offset between the PU prediction block and the reference location.
[0067] [0067] When using bipredition to encode a PU, video encoder 20 can determine a first reference location in a reference image in RefPicList0 and a second reference location in a reference image in RefPicList1. The video encoder 20 can then generate, based at least in part on samples corresponding to the first and second reference sites, the predictive blocks for the PU. In addition, when using bipredition to encode the PU, the video encoder 20 can generate a first motion vector that indicates a spatial displacement between a sample block of the PU and the first reference location and a second motion vector that indicates a spatial shift between the PU prediction block and the second reference site.
[0068] [0068] In some examples, JEM / VVC also provides a related motion compensation mode, which can be considered an inter-prediction mode. In the related motion compensation mode, the video encoder 20 can determine two or more motion vectors representing non-translational motion, such as zooming in or out, rotation, perspective movement or other types of irregular motion.
[0069] [0069] After the video encoder 20 generates predictive luma, Cb, and Cr blocks for one or more PUs of a CU, the video encoder 20 can generate a residual luma block for the CU. Each sample in the CU luma residual block indicates a difference between a luma sample in one of CU's predictive luma blocks and a corresponding sample in the original CU luma coding block. In addition, video encoder 20 can generate a residual Block Cb for the CU. Each sample in the CU residual block Cb can indicate a difference between a Cb sample in one of CU's predictive Cb blocks and a corresponding sample in the original CU Cb coding block. The video encoder 20 can also generate a residual block Cr for the CU. Each sample in the residual CU block of the CU can indicate a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original CU coding block.
[0070] [0070] In addition, video encoder 20 can use quadratic tree partitioning to decompose the residual blocks luma, Cb and, Cr of a CU into one or more blocks of luma transform, Cb, and Cr. A transform block can be a rectangular sample block to which the same transform is applied. A transform unit (TU) of a CU can comprise a luma sample transform block, two corresponding chroma sample transform blocks, and structures and syntax used to transform the transform block samples. In a monochrome image or an image that has three separate color planes, a TU can comprise a single transform block and structures and syntax used to transform the samples in the transform block. Thus, each TU of a CU can be associated with a luma transform block, a Cb transform block, and a Cr transform block. The luma transform block associated with the TU can be a sub-block of the luma residual block of the CU. The transform block Cb can be a sub-block of the residual block Cb of CU. The transform block Cr can be a sub-block of the residual block Cr of CU.
[0071] [0071] The video encoder 20 can apply one or more transforms to a luma transform block of a TU to generate a lum coefficient block for the TU. A coefficient block can be a two-dimensional matrix of transform coefficients. A transform coefficient can be a scalar quantity. The video encoder 20 can apply one or more transforms to a transform block Cb of a TU to generate a block of coefficient Cb for the TU. The video encoder 20 can apply one or more transforms to a TU Cr transform block to generate a Cr coefficient block for the TU.
[0072] [0072] After generating a coefficient block (for example, a luma coefficient block, a Cb coefficient block or a Cr coefficient block), the video encoder 20 can quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, providing additional compression. In addition, video encoder 20 can inversely quantize transform coefficients and apply an inverse transform to the transform coefficients in order to reconstruct transform blocks of TUs from CUs in an image. The video encoder 20 can use the CU's reconstructed TU transform blocks and the CU's PU predictive blocks to reconstruct CU's encoding blocks. By reconstructing the encoding blocks for each CU of an image, video encoder 20 can reconstruct the image. Video encoder 20 can store reconstructed images in a decoded image buffer (DPB). The video encoder 20 can use reconstructed images in the DPB for inter-prediction and intra-prediction.
[0073] [0073] After the video encoder 20 quantizes a coefficient block, the video encoder 20 can entropy encode syntax elements that indicate the quantized transform coefficients. For example, video encoder 20 can perform binary arithmetic encoding adaptive to the context
[0074] [0074] The video encoder 20 can output a bit stream that includes a bit stream that forms a representation of encoded images and associated data. The bit stream may comprise a sequence of network abstraction layer (NAL) units. Each NAL unit includes an NAL unit header and encapsulates a gross byte sequence payload (RBSP). The NAL unit header can include a syntax element that indicates an NAL unit type code. The NAL unit type code specified by the NAL unit header of an NAL unit indicates the type of the NAL unit. An RBSP can be a syntax structure that contains an integer number of bytes that is encapsulated within an NAL unit. In some cases, an RBSP includes zero bits.
[0075] [0075] Different types of NAL units can encapsulate different types of RBSPs. For example, a first type of NAL unit can encapsulate an RBSP for a set of image parameters (PPS), a second type of NAL unit can encapsulate an RBSP for an encoded slice, a third type of NAL unit can encapsulate an RBSP to supplement enhancement information (SEI), and so on. A PPS is a syntax structure that can contain elements of syntax that apply to zero or more entire encoded images. NAL units that encapsulate RBSPs for video encoding data (as opposed to RBSPs for parameter sets and SEI messages) can be called NAL units of the video encoding layer (VCL). An NAL unit that encapsulates an encoded slice may be referred to herein as an NAL unit of encoded slice. An RBSP for an encoded slice can include a slice header and slice data.
[0076] [0076] The video decoder 30 can receive a bit stream. In addition, video decoder 30 can analyze the bit stream to decode syntax elements from the bit stream. The video decoder 30 can reconstruct the images of the video data based at least in part on the syntax elements decoded from the bit stream. The process for reconstructing the video data can generally be reciprocal to the process performed by the video encoder 20. For example, the video decoder 30 can use PU motion vectors to determine predictive blocks for the PUs of a current CU. The video decoder 30 can use a motion vector or motion vectors from PUs to generate predictive blocks for the PUs.
[0077] [0077] In addition, the video decoder 30 can inversely quantize blocks of coefficients associated with TUs of the current CU. The video decoder 30 can perform a reverse transform on the coefficient blocks to reconstruct transform blocks associated with the TUs of the current CU. The video decoder 30 can reconstruct the encoding blocks of the current CU by adding samples from the predictive sample blocks for PUs of the current CU to corresponding samples of the transform blocks of the current CU's TUs. By reconstructing the encoding blocks for each CU of an image, the video decoder 30 can reconstruct the image. The video decoder 30 can store decoded images in a decoded image buffer for output and / or for use in decoding other images.
[0078] [0078] Next generation video applications are expected to operate with video data representing scenarios captured with an HDR and WCG. The parameters of the dynamic range used and the color range are two independent attributes of video content, and their specifications for the purposes of digital television and multimedia services are defined by various international standards. For example, ITU-R Rec. BT.709, “Parameter values for HDTV standards for international production and exchange of programs”, defines parameters for HDTV (high definition television), as standard dynamic range (SDR) and standard color range. ITU-R Rec. BT.2020, “Parameter values for high definition television systems for the production and international exchange of programs”, specifies UHDTV (high definition television), such as HDR and WCG (for example, primary definition colors WCG that extend beyond the standard gamma color). Recording. BT.2100, “Image parameter values for high dynamic range television for use in production and international program exchange” defines transfer functions and representations for use on HDR television, including primaries that support representations of a wide range of colors. There are also other documents from the standards development organization (SDOs) that specify attributes of dynamic range and color gamut in other systems, for example, the DCI-P3 color gamut is defined in SMPTE-231-2 (Society of Engineering Engineers) Cinema and Television) and some HDR parameters are defined in the SMPTE-2084. A brief description of the dynamic range and color range for video data is provided below.
[0079] [0079] The dynamic range is typically defined as the ratio between the maximum and minimum brightness (for example, luminance) of the video signal. The dynamic range can also be measured in terms of “f-stop”, where an f-stop corresponds to a doubling of the dynamic range of a signal. In the MPEG definition, content that varies in brightness with more than 16 f-stops is called HDR content. In some terms, levels between 10 and 16 stops are considered as intermediate dynamic range, but can be considered HDR in other definitions. In some examples of this disclosure, HDR video content can be any video content that has a higher dynamic range than video content traditionally used with a standard dynamic range (for example, video content as specified by ITU-R Rec. BT.709).
[0080] [0080] The human visual system (HVS) is capable of perceiving dynamic ranges much larger than the SDR and HDR content. However, the HVS includes an adaptation mechanism to restrict the dynamic range of the HVS to a so-called simultaneous range. The width of the simultaneous strip may depend on the current lighting conditions (for example, current brightness). A view of the dynamic range provided by the HDTV SDR, the expected HDR from UHDTV and the dynamic range from HVS is shown in Figure 3, although the exact range may vary based on each individual and display.
[0081] [0081] Some sample video applications and services are regulated by ITU Rec.709 and provide an SDR, generally supporting a brightness range (for example, luminance) of about 0.1 to 100 candelas (cd) per m2 ( usually called “nits”), leading to less than 10 f-stops. Some examples of next generation video services are expected to provide dynamic range of up to 16 f-stops. Although detailed specifications for this content are currently under development, some initial parameters have been specified in SMPTE2084 and ITU-R Rec. 2020.
[0082] [0082] Another aspect for a more realistic video experience, in addition to HDR, is the color dimension. The color dimension is usually defined by the color gamut. Figure 4 is a conceptual diagram showing an SDR color gamut (triangle 100 based on BT.709 color primers) and a wider color gamut than for UHDTV (triangle 102 based on BT.2020 color primaries. ). Figure 4 also represents the so-called spectrum location (delimited by the 104-shaped area), representing the limits of natural colors. As shown in Figure 4, moving from the BT.709 (triangle 100) color primaries to BT.2020 (triangle
[0083] [0083] Examples of color gamut specifications for the DCI-P3, BT.709, and BT.2020 color spaces are shown in Table 1. Table 1 - Color Gamma Parameters RGB color space parameters Primary colors of white color XXw yyw XXR yyR XXG yyG XXB yyB DCI-P3 0.314 0.351 0.680 0.320 0.265 0.690 0.150 0.060 ITU-R 0.3127 0.3290 0.64 0.33 0.30 0.60 0.15 0, 06 BT.709 ITU-R 0.3127 0.3290 0.708 0.292 0.170 0.777 0.131 0.046 BT.2020
[0084] [0084] As can be seen in Table 1, a color range can be defined by the X and Y values of a white dot and the x and y values of the primary colors (for example, red (R), green (G) and blue ( B) The x and y values represent normalized values derived from the chromaticity (X and Z) and the brightness (Y) of the colors, as defined by the CIE 1931 color space. The CIE 1931 The color space defines the links between pure colors ( for example, in terms of wavelength) and how the human eye perceives these colors.
[0085] [0085] HDR / WCG video data is typically acquired and stored with very high accuracy per component (same floating point), with a 4: 4: 4 chroma format and a very wide color space (for example, CIE XYZ). This representation targets high precision and is almost mathematically lossless. However, this format for storing HDR / WCG video data can include many redundancies and may not be ideal for compression purposes. A less accurate format with HVS-based assumptions is typically used for high-end video applications.
[0086] [0086] An example of a video data format conversion process for compression purposes includes three main processes, as shown in Figure
[0087] [0087] The reverse conversion on the decoder side is shown in Figure 6. The techniques in Figure 6 can be performed by the target device 14. The converted HDR data 120 can be obtained on the target device 14 by decoding data from video using a hybrid video decoder (for example, video decoder 30 using H.264, HEVC or VVC techniques). The HDR '120 data can then be inversely quantized by the inverse quantization unit 122. Then, an inverse color conversion process 124 can be applied to the inverse quantized HDR' data. The reverse color conversion process 124 can be the reverse color conversion process 114. For example, the reverse color conversion process 124 can convert HDR 'data from a YCrCb format back to an RGB format. Then, the inverse transfer function 126 can be applied to the data to re-add the dynamic range that has been compressed by the transfer function 112 to recreate the linear RGB data 128.
[0088] [0088] The techniques represented in Figure 5 will now be discussed in more detail. In general, a transfer function is applied to the data (for example, HDR / WCG video data) to compress the dynamic range of the data, so that errors due to quantization are perceptibly uniform (approximately) over the entire range of data. luminance values. This compression allows data to be represented with fewer bits. In one example, the transfer function may be a one-dimensional non-linear (1D) function and may reflect the reverse of an electro-optical transfer (EOTF) function on the end user's display, for example, as specified for SDR in Rec.
[0089] [0089] The SMPTE-2084 specification defined the EOTF application as follows. The transfer function is applied to the normalized linear values of R, G, B, which results in a non-linear representation of R'G'B '. SMPTE-2084 defines normalization by NORM = 10000, which is associated with a maximum brightness of 10000 nits (cd / m2).
[0090] [0090] With the input values (linear color value) normalized to the range 0..1, the normalized output values (non-linear color value) of PQ EOTF are shown in Figure 8. As seen in curve 131 , 1% (low light) of the dynamic range of the input signal is converted to 50% of the dynamic range of the output signal.
[0091] [0091] Typically, an EOTF is defined as a function with floating point precision; therefore, no error is introduced in a signal with this non-linearity if the inverse TF (the so-called OETF) is applied. The inverse TF (OETF) specified in SMPTE-2084 is defined as an inversePQ function: With
[0092] [0092] With floating point precision, the sequential application of an EOTF and OETF provides a perfect reconstruction without errors. However, this representation is not always ideal for streaming or transmission services. A more compact representation with fixed bit precision of nonlinear R'G'B 'data is described in the following sections.
[0093] [0093] Note that currently EOTF and OETF are subject to very active research, and the TF used in some HDR video encoding systems may differ from SMPTE2084.
[0094] [0094] In the context of this disclosure, the terms “signal value” or “color value” can be used to describe a level of luminance corresponding to the value of a specific color component (such as R, G, B or Y) for an image element. The signal value is typically representative of a linear light level (luminance value). The terms “code level” or “digital code value” can refer to a digital representation of an image signal value. Typically, this digital representation is representative of a non-linear signal value. An EOTF represents the relationship between the non-linear signal values supplied to a display device (for example, display device 32) and the linear color values produced by the display device.
[0095] [0095] RGB data is generally used as input color space, since RGB is the type of data that is normally produced by image capture sensors. However, the RGB color space has high redundancy between its components and is not ideal for compact representation. To obtain a more compact and more robust representation, RGB components are usually converted (for example, a color transformation is performed) into an uncorrelated color space that is more suitable for compression, for example, YCbCr. A YCbCr color space separates brightness in the form of luminance (Y) and color information (CrCb) into different, less correlated components. In this context, a robust representation can refer to a color space with higher levels of error resilience when compressed with a restricted bit rate.
[0096] [0096] Modern video encoding systems typically use the YCbCr color space, as specified in ITU-R BT.709 or ITU-R BT.709. The YCbCr color space in the BT.709 standard specifies the following conversion process from R’G’B ’to Y’CbCr (non-constant luminance representation):
[0097] [0097] The above can also be implemented using the following approximate conversion that avoids the division of the Cb and Cr components:
[0098] [0098] The ITU-R BT.2020 standard specifies the following conversion process from R’G’B ’to Y’CbCr (non-constant luminance representation):
[0099] [0099] The above can also be implemented using the following approximate conversion that avoids the division of the Cb and Cr components:
[0100] [0100] Note that both color spaces remain normalized. Therefore, for the normalized input values in the range 0 ... 1, the resulting values will be mapped to the range 0..1. Generally, color transformation implemented with floating point precision provides perfect reconstruction; therefore, this process is lossless.
[0101] [0101] After color transformation, the input data in a target color space can still be represented with a high bit depth (for example, floating point precision). That is,
[0102] [0102] Certain studies show that the precision of 10 to 12 bits in combination with the transfer of PQ is sufficient to provide HDR data of 16 f-stops with distortion below the Justifiable Difference (JND). In general, a JND is the amount of something (for example, video data) that must be changed in order for a difference to be noticeable (for example, by the HVS). Data represented with 10-bit precision can be further encoded with most cutting-edge video encoding solutions. This conversion process includes signal quantization and is a lossy coding element and is a source of inaccuracy introduced in the converted data.
[0103] [0103] An example of this quantization applied to code words in the target color space. In this example, the YCbCr color space is shown below. Input values YCbCr represented with floating point precision are converted into a BitDepthY fixed bit depth signal to the value Y
[0104] [0104] Next generation HDR / WCG video applications are expected to operate with video data captured in different HDR and CG parameters. Examples of different configurations may be capturing HDR video content with a maximum brightness of up to 1000 nits or up to 10,000 nits. Examples of different color ranges can include BT.709, BT.2020, as well as specified SMPTE P3 or others.
[0105] [0105] It is also envisaged that a single color space, for example, a target color container, which incorporates (or almost incorporates) all the other color ranges currently used for use in the future. Examples of such a target color container include BT.2020 and BT.2100. Support for a single destination color container would significantly simplify the standardization, implementation and deployment of HDR / WCG systems, as there were a reduced number of operational points (for example, number of color containers, color spaces, algorithms for color conversion, etc.) and / or a reduced number of necessary algorithms must be supported by a decoder (eg video decoder 30).
[0106] [0106] In an example of such a system, content captured with a native color gamut (for example, P3 or BT.709) other than the destination color container (for example, BT.2020) can be converted into the destination before processing (for example, before video encoding by video encoder 20).
[0107] [0107] During this conversion, the range of values occupied by each component (eg RGB, YUV, YCrCb, etc.) of a signal captured in the P3 or BT.709 color range can be reduced in a BT.2020 representation . Since the data is represented with floating point precision, there is no loss; however, when combined with color conversion and quantization, shrinking the range of values leads to an increase in the quantization error of the input data.
[0108] [0108] In addition, in a real-world encoding system, encoding a signal with reduced dynamic range can lead to a significant loss of accuracy for encoded chroma components and would be observed by a viewer as encoding artifacts, for example, color mismatch and / or color leak.
[0109] [0109] In addition, some of the non-linearities (for example, introduced by the transfer function of the SMPTE2084) and color representations (for example, ITU-R BT.2020 or BT.22100) used in modern video encoding systems can result in a video data representation that presents significant variation of the perceived distortion, or perceptible difference threshold (JND), on the dynamic range and the color components of the signal representation. This can be called perceived distortion. For such representations, a quantization scheme that applies a uniform scalar quantizer over the dynamic range of luma or chroma values can introduce a quantization error with a different merit of perception, depending on the magnitude of the quantized samples. This impact on signals can be interpreted as a processing system with non-uniform quantization that results in unequal signal-to-noise ratios within the range of processed data.
[0110] [0110] An example of this representation is a video signal represented in the YCbCr color space of non-constant Luminance (NCL) whose color primaries are defined in the ITU-R register. BT.2020, with transfer function SMPTE-2084, or BT.2100. As shown in Table 1 above, this representation allocates significantly more code words for low signal strength values compared to the number of code words used in mid-range values (e.g., 30% of code represent linear light samples <10 nits with only
[0111] [0111] Another example of this representation is the effective dynamic range of the chroma components of the ITU-R BT.2020 / BT.2100 color representations. These containers support a much wider range of colors than conventional color containers (for example, BT.709). However, under the fine bit depth restrictions used in a specific representation (for example, 10 bits per sample), the finite bit depth would effectively reduce the granularity of conventional content (for example, with the BT.709 range) in new format compared to using a native container (eg BT.709).
[0112] [0112] This situation is shown in Figures 9A and 9B, where the colors of the HDR sequences are represented in an xy color plane. Figure 9A shows colors from a “Tibul” test sequence captured in the native BT.709 color space (triangle 150). However, the colors in the test sequence (shown as dots) do not occupy the full color range of the BT.709 or BT.2020. In
[0113] [0113] As a result, a video encoding system (for example, H.265 / HEVC) featuring uniform quantization for all the range and color components of the data would introduce much more visually perceived distortion in the chroma components of a signal in one color BT.2020 container compared to a sign in a BT.709 color container.
[0114] [0114] Yet another example of a significant variation in perceived distortion, or the perceptible difference limit (JND), above the dynamic range or color components of some color representations, are color spaces resulting from the application of an Adjustment Dynamic Range (DRA), or remodeler. DRA can be applied to video data partitioned into dynamic range partitions, with the aim of providing a uniform perception of distortion within a full dynamic range. However, applying DRA to a color component independently can, in turn, introduce a bit rate reallocation from one color component to another (for example, a higher bit budget spent on luma components can result in more distortion) introduced in chroma components and vice versa).
[0115] [0115] Other techniques to solve the problem of non-uniform distribution of perceptual quality code words in the representation of high-end colors, such as BT.2020 or BT.2100, have been proposed.
[0116] [0116] "Dynamic Range Adjustment SEI to enable high dynamic range video encoding with backward compatible capability", D. Rusanovskyy, AK Ramasubramonian, D. Bugdayci, S. Lee, J. Sole, M. Karczewicz, document VCEG COM16-C 1027-E, September 2015, describes the application of a Dynamic Range Adjustment (DRA) to obtain code word redistribution in video data in the SMPTE-2084 / BT.2020 color container before applying a transformation-based hybrid video encoding scheme (for example, H.265 / HEVC).
[0117] [0117] The redistribution obtained by the application of a DRA aims at the linearization of the perceived distortion (for example, signal / noise ratio) within a dynamic range. To compensate for this redistribution on the decoder side and convert data to the original ST2084 / BT.2020 representation, a reverse DRA process is applied to the data after the video decoding.
[0118] [0118] In one example, DRA can be implemented as a very targeted linear function f (x), which is defined for a group of non-overlapping dynamic range partitions (ranges) {Ri} of the input value x, where i it is an index of the range. The term i = 0 ... N-1, where N is the total number of intervals {Ri} used to define the DRA function. Suppose the DRA intervals are defined by a minimum and maximum value of x that belongs to the range Ri, for example [xi, xi + 1 - 1], where xi and x1 + 1 indicate the minimum value of the intervals Ri and Ri + 1, respectively. When applied to the color component Y (luma) of the video data, the DRA Sy function is defined using a scale Sy 1 and a value of 0y, i, which are applied to all x Є [xi, xi + 1 - 1], so Sy = {Sy, i, 0y, i}.
[0119] [0119] Thus, for any Ri and all x Є [xi, xi + 1 - 1], the output value X is calculated as follows:
[0120] [0120] For the inverse DRA mapping process for the luma Y component performed on the decoder, the DRA Sy function is defined by inverse scale values Sy, ie displacement values 0y, i, which are applied to all X Є [xi , xi + 1 - 1].
[0121] [0121] With this, for any Ri and all, and every X Є [xi, xi + 1 - 1], the reconstructed value x is calculated as follows:
[0122] [0122] The process of direct mapping of the DRA to the chroma components Cb and Cr can be defined as follows. This example is given with the sample that indicates the term u denoting a sample of the color component Cb, which belongs to the range Ri, u Є [ui, ui + 1 - 1], so Su = {Su, i, 0u, i} : where Offset equal to 2 (bitdepth-1) denotes the bipolar shift of the signal Cb, Cr.
[0123] [0123] The reverse DRA mapping process conducted in the decoder for chroma components Cb and Cr can be defined as follows. An example is given with the sample that indicates the term U of the remapped Cb color component that belongs to the range Ri, U Є [Ui, Ui + 1 - 1]: where Offset equal to 2 (bitdepth-1) denotes the bipolar shift of the sign Cb, Cr.
[0124] [0124] “Performance investigation of high dynamic range and wide color gamut video coding techniques”, J. Zhao, S.-H. Kim, A. Segall, K. Misra, VCEG document COM16-C 1030-E, September 2015, describes a spatially intensive (block-based) quantization scheme to align bit rate allocation and distortion visually perceived between the video encoding applied in Y2020 (Representations ST2084 / BT2020) and Y709 (BT1886 / BT2020). It was observed that, to maintain the same level of quantization for luma components, the quantization of the signal in Y2020 and Y709 may differ by a value that depends on the luma, so that:
[0125] [0125] The function f (Y2020) was linear for the intensity values (brightness level) of the video in Y2020, and can be approximated as:
[0126] [0126] The proposed spatially variable quantization scheme being introduced in the coding stage showed to be able to improve the signal-quantization noise rate visually perceived for the video signal encoded in the ST2084 / BT.2020 representations. The delta QP application and QP derivation mechanism for chroma components allows the propagation of luma deltaQP to chroma components, thus allowing some degree of compensation for chroma components due to the increased bit rate introduced in the luma component.
[0127] [0127] “Luma-driven chroma scaling (LCS) design, HDR CE2: Report on CE2.a-1 LCS”, JCTVC- W0101, A.K. Ramasubramonian, J. Sole, D. Rusanovskyy, D. Bugdayci, M. Karczewicz, describes a method for adjusting chroma information (for example, Cb and Cr components), exploring the brightness information associated with the processed chroma sample. Similar to the DRA method above, it has been proposed to apply a scale factor Su to a chroma sample Cb and to apply a scale factor S u, i to a chroma sample Cr. However, instead of defining a DRA function as a highly targeted linear function Su = {Su, i, 0u, i} for a set of bands {Ri} accessible by the chroma value u or v as in Equations (9) and (10), an example LCS method uses the luma Y value to derive a scale factor for the chroma sample. Thus, the LCS mapping directly from the u (or v) chroma sample is conducted as:
[0128] [0128] The reverse LCS process conducted on the decoder side is defined as follows:
[0129] [0129] In more detail, for a given pixel located at (x, y), the chroma samples Cb (x, y) or Cr (x, y) are sized with a factor derived from their LCS function S cb or Scr, which is accessed by the corresponding luma value Y '(x, y).
[0130] [0130] In the advanced LCS for chroma samples, the values of Cb (or Cr) and the associated luma values Y 'are taken as input for the chroma scale function Scb (or Scr) and are converted to Cb' and Cr ', as shown in Equation 15. On the decoder side, the reverse LCS is applied. The reconstructed Cb 'or Cr' are converted to Cb, or Cr, as shown in Equation (16).
[0131] [0131] Figure 10 shows an example of an LCS function, where the chroma scale factor is a function of the associated luma value. In the example in Figure 10, using the LCS 153 function, chroma pixel components with lower luma values are multiplied with smaller scale factors.
[0132] [0132] This disclosure describes techniques, methods and apparatus for performing dynamic range adjustment (DRA), including cross-component DRA to compensate for dynamic range changes introduced in HDR signal representations by color gamma conversion. Adjusting the dynamic range can help prevent and / or decrease any distortion caused by a color gamut conversion, including color mismatch, color leakage, etc. In one or more examples of the disclosure, the DRA is performed on the values of each color component of the target color space, for example, YCbCr, before quantization on the encoder side (for example, by the video preprocessor unit 19 of the source device 12) and after inverse quantization on the decoder side (for example, by the video post-processor unit 31 of the target device 14). In other examples, the DRA techniques of this disclosure can be performed on the video encoder 20 and video decoder 30.
[0133] [0133] Figure 11 is a block diagram illustrating an example HDR / WCG conversion device that operates according to the techniques of this disclosure. In FIG. 11, solid lines specify the data flow and dashed lines specify control signals. The techniques of this disclosure can be performed by the video preprocessor unit 19 of the source device 12 or by the video encoder 20. As discussed above, the video preprocessor unit 19 can be a device separate from the video encoder. 20. In other examples, the video preprocessor unit 19 can be incorporated into the same device as the video encoder 20.
[0134] [0134] As shown in Figure 11, the RGB 200 native CG video data is input to the video preprocessor unit 19. In the context of video preprocessing by the video preprocessor unit 19, the RGB 200 native CG video data is defined by an input color container. The input color container specifies the set of primary colors used to represent video data 200 (for example, BT.709, BT.2020, BT.2100 P3, etc.). In an example of the disclosure, the video preprocessor unit 19 can be configured to convert the color container and color space of the native CB video data from RGB 200 into a target color container and a color space. target for HDR '216 data. The input color container, the target color container can specify a set or primary colors used to represent HDR' 216 data. In an example of the disclosure, video data from Native RGB 200 CBs can be HDR / WCG videos and can have a BT. The 2020 or P3 color container (or any WCG) and is in an RGB color space. In another example, the RGB 200 native CB video data can be SDR video and can have a BT.709 color container. In one example, the destination color container for HDR data '216 may have been configured for HDR / WCG video (for example, BT.2020 color container) and may use a more ideal color space for color coding. video (for example, YCrCb).
[0135] [0135] In an example of the disclosure, the CG 202 converter can be configured to convert the color container of the RGB 200 native CG video data from the input color container (eg, first color container) to the container destination colors (for example, second container color). As an example, the CG 202 converter can convert native RGB video data from RGB 200 from a BT.709 color representation to a BT.2020 color representation, an example of which is shown below.
[0136] [0136] The process to convert samples from RGB BT.709 (R709, G709, B709) to RGB BT.2020 samples (R2020, G2020, B2020) can be implemented with a two-step conversion that involves converting first into the XYZ representation , followed by a conversion from XYZ to RGB BT.2020 using the appropriate conversion matrices.
[0137] [0137] Conversion from XYZ to R2020G2020B2020 (BT.2020):
[0138] [0138] The resulting video data after CG conversion is shown as RGB 204 target CG video data in Figure 11. In other examples of the disclosure, the color container for the input data and the output HDR data may be the same. In this example, the CG converter 202 does not need to perform any conversion on the RGB native CG video data
[0139] [0139] The transfer function unit 206 then compresses the dynamic range of the RGB target CG video data 204. The transfer function unit 206 can be configured to apply a transfer function to compress the dynamic range in the same way as discussed above with reference to Figure 5. The color conversion unit 208 converts the target CG color data from RGB 204 from the color space of the input color container (for example, RGB) to the color space of the target color container (for example, YCrCb). As explained above with reference to Figure 5, the color conversion unit 208 converts the compressed data into a more compact or robust color space (for example, a YUV or YCrCb color space) that is more suitable for compression by an encoder hybrid video (for example, video encoder 20).
[0140] [0140] The adjustment unit 210 is configured to perform a dynamic range adjustment (DRA) of the video data converted to color according to the DRA parameters derived from the DRA 212 parameter estimation unit. In general, after the CG conversion by CG converter 202 and compression of the dynamic range by transfer function unit 206, the actual color values of the resulting video data may not use all available code words (for example, unique bit strings representing each color)
[0141] [0141] Adjustment unit 210 can be configured to apply DRA parameters to the color components (eg, YCrCb) of the video data, for example, RGB 204 target CG video data after dynamic range compression and color conversion to make full use of available code words for a specific target color container. Adjustment unit 210 can apply the DRA parameter to video data at a pixel level (for example, for each color component of a pixel). In general, DRA parameters define a function that expands the code words used to represent the actual video data to as many code words as possible for the target color container.
[0142] [0142] In an example of disclosure, DRA parameters include a scale and offset value that is applied to the color components of the video data (for example, luma (Y) and chroma (Cr, Cb) color components) . The offset parameter can be used to center the values of the color components in the center of the available code words for a target color container. For example, if a target color container includes 1024 code words per color component, an offset value can be chosen so that the central code word is moved to code word 512 (for example, the most central code). In other examples, the offset parameter can be used to provide better mapping of input codewords to output codewords, so that the overall representation in the destination color container is more efficient in combating coding artifacts.
[0143] [0143] In one example, adjustment unit 210 applies DRA parameters to video data in the target color space (for example, YCrCb) as follows: where the signal components Y ', Cb' and Cr 'are one signal produced from the conversion from RGB to YCbCr (example in equation 3). Note that Y ', Cr' and Cr 'can also be a video signal decoded by the video decoder 30. Y ”, Cb” and Cr ”are the color components of the video signal after the DRA parameters have been applied to each color component.
[0144] [0144] As can be seen in the example above, each color component is related to different scale and offset parameters. For example, scale1 and offset1 are used for component Y ', scale2 and offset2 are used for component Cb', and scale3 and offset3 are used for component Cr '. It should be understood that this is just an example. In other examples, the same scale and offset values can be used for each color component. As will be explained in more detail below, the DRA 312 parameter derivation unit can be configured for certain scale values of the chroma components (for example, Cr, Cb) as a function of the scale values determined for the luma components.
[0145] [0145] In other examples, each color component can be associated with various scale and offset parameters. For example, the actual distribution of chroma values for the Cr or Cb color components may differ for different partitions or code word ranges. As an example, there may be more unique codewords used above the central codeword (for example, code word 512) than those below the central codeword. In this example, adjustment unit 210 can be configured to apply a set of scale and offset parameters to chroma values above the central codeword (for example, having values greater than the central codeword) and apply a different set from scale and offset parameters to chroma values below the central code word (for example, having values less than the central code word).
[0146] [0146] As can be seen in the example above, adjustment unit 210 applies the DRA scale and offset parameters as a linear function. As such, it is not necessary for the adjustment unit 210 to apply the DRA parameters to the target color space after the color conversion by the color conversion unit 208. This is because the color conversion itself is a linear process. As such, in other examples, adjustment unit 210 can apply DRA parameters to video data in the native color space (e.g., RGB) prior to any color conversion process. In this example, color conversion unit 208 would apply color conversion after adjustment unit 210 applied DRA parameters.
[0147] [0147] In another example of the disclosure, the adjustment unit 210 can apply the DRA parameters in the target color space or in the native color space as follows: In this example, the parameter scale1, scale2, scale3, offset1, offset2 , and offset3 have the same meaning as described above. The offsetY parameter is a parameter that reflects the signal brightness and can be equal to the average value of Y ’. In other examples, an offset parameter similar to offsetY can be applied to components Cb 'and Cr' to better preserve the mapping of the central value in the input and output representations.
[0148] [0148] In another example of the disclosure, the adjustment unit 210 can be configured to apply the DRA parameters to a color space except the native color space or the target color space. In general, adjustment unit 210 can be configured to apply DRA parameters as follows: where signal components A, B and C are signal components in a color space that is different from the target color space, for example , RGB or an intermediate color space.
[0149] [0149] In other examples of the disclosure, the adjustment unit 210 is configured to apply a linear transfer function to the video to perform DRA. Such a transfer function is different from the transfer function used by transfer function unit 206 to compress the dynamic range. Similar to the scale and offset terms defined above, the transfer function applied by the adjustment unit 210 can be used to expand and center the color values in the code words available in a destination color container. An example of applying a transfer function to perform the DRA is shown below: The term TF2 specifies the transfer function applied by adjustment unit 210. In some examples, adjustment unit 210 can be configured to apply different transfer functions to each of the components.
[0150] [0150] In another example of the disclosure, the adjustment unit 210 can be configured to apply DRA parameters evenly with the color conversion of the color conversion unit 208 in a single process. That is, the linear functions of the adjustment unit 210 and the color conversion unit 208 can be combined. An example of a combined application, where f1 and f2 are a combination of the RGB matrix for YCbCr and the DRA scale factors, is shown below:
[0151] [0151] In another example of the disclosure, after the application of the DRA parameters, the adjustment unit 210 can be configured to perform a sending process to prevent the video data from having values outside the range of code words specified for a specific destination color container. In some circumstances, the scale and offset parameters applied by the adjustment unit 210 may cause some color component values to exceed the range of allowed code words. In this case, the adjustment unit 210 can be configured to cut component values that exceed the range to the maximum value in the range.
[0152] [0152] The DRA parameters applied by adjustment unit 210 can be determined by the DRA parameter estimation unit 212. The frequency and time periods in which the DRA parameter estimation unit 212 updates the DRA parameters are flexible. For example, the DRA 212 parameter estimation unit can update the DRA parameters on a temporal level. That is, new DRA parameters can be determined for a group of images (GOP) or a single image (frame). In this example, the RGB 200 native CG video data can be a GOP or a single image. In other examples, the DRA parameter estimation unit 212 can update the DRA parameters at a spatial level, for example, at the slice block or at the block level. In this context, a video data block can be a macroblock, encoding tree unit (CTU), encoding unit or any other block size and shape. A block can be square, rectangular or any other shape. Therefore, DRA parameters can be used for more efficient temporal and spatial prediction and coding.
[0153] [0153] In an example of the disclosure, the DRA 212 parameter estimation unit can derive the DRA parameters based on the matching of the native color gamut of the RGB 200 native CG video data and the color gamut of the container of target colors. For example, the DRA 212 parameter estimation unit can use a set of predefined rules to determine scale and offset values, given a certain native color range (for example, BT.709) and the color range of a container target colors (for example, BT.2020).
[0154] [0154] In some examples, the DRA 212 parameter estimation unit can be configured to estimate DRA parameters by determining the primary coordinates in primeN from the actual distribution of the color values in the RGB 200 native CG video data , not the predefined primary values of the native color gamut. That is, the DRA 212 parameter estimation unit can be configured to analyze the actual colors present in the RGB 200 native CG video data and use the primary color values and white point determined from this analysis in the described function. above to calculate the DRA parameters. The approximation of some parameters defined above can be used as DRA to facilitate the calculation.
[0155] [0155] In other examples of the disclosure, the DRA 212 parameter estimation unit can be configured to determine the DRA parameters based not only on the color gamut of the target color container, but also on the target color space. Actual value distributions of component values may differ from one space to another. For example, the chroma value distributions may be different for YCbCr color spaces with a constant luminance compared to YCbCr color spaces with a non-constant luminance. The DRA 212 parameter estimation unit can use the color distributions of different color spaces to determine the DRA parameters.
[0156] [0156] In other examples of the disclosure, the DRA 212 parameter estimation unit can be configured to derive values for DRA parameters, in order to minimize certain cost functions associated with pre-processing and / or encoding video data . As an example, the DRA parameter estimation unit 212 can be configured to estimate DRA parameters that minimized quantization errors introduced by the 214 quantization unit. The DRA parameter estimation unit 212 can minimize this error by performing error tests. of quantization in video data that had different sets of DRA parameters applied. In another example, the DRA parameter estimation unit 212 can be configured to estimate DRA parameters that perceptually minimize the quantization errors introduced by the quantization unit 214. The DRA 212 parameter estimation unit can minimize this error based on perceptual error tests on video data that have different sets of DRA parameters applied. The DRA 212 parameter estimation unit can then select the DRA parameters that produced the least quantization error.
[0157] [0157] In another example, the DRA parameter estimation unit 212 can select DRA parameters that minimize a cost function associated with the DRA performed by the adjustment unit 210 and the video encoding performed by the video encoder
[0158] [0158] In each of the above techniques for DRA parameter estimation, the DRA parameter estimation unit 212 can determine the DRA parameters separately for each component using information regarding the component. In other examples, the DRA 212 parameter estimation unit can determine the DRA parameters using cross-component information. For example, as will be discussed in more detail below, DRA parameters (eg scale) derived for a luma (Y) component can be used to derive DRA parameters (eg scale) for chroma component (Cr and / or Cb).
[0159] [0159] In addition to deriving DRA parameters, the DRA 212 parameter estimation unit can be configured to signal DRA parameters in an encoded bit stream. The DRA parameter estimation unit 212 can signal one or more syntax elements that directly indicate the DRA parameters, or can be configured to provide the one or more syntax elements to the video encoder 20 for signaling. Such syntax elements of the parameters can be signaled in the bit stream so that the video decoder 30 and / or post-
[0160] [0160] In one example, the DRA 212 and / or video encoder 20 parameter estimation unit can signal one or more syntax elements in a video bit stream encoded as metadata, in a further enhancement information message (SEI), in video usability information (VUI), in a set of video parameters (VPS), in a set of sequence parameters (SPS), in a set of image parameters, in a slice header, in a CTU header or any other suitable syntax structure to indicate the DRA parameters for the size of the video data (for example, GOP, images, blocks, macroblock, CTUs, etc.).
[0161] [0161] In some examples, the one or more syntax elements explicitly indicate DRA parameters. For example, the one or more syntax elements can be the various scale and offset values for DRA. In other examples, the one or more elements of syntax can be one or more indexes in a lookup table that includes scale and offset values for DRA. In yet another example, the one or more syntax elements can be indices in a lookup table that specify the linear transfer function to be used for DRA.
[0162] [0162] In other examples, the parameters of
[0163] [0163] After adjustment unit 210 applies DRA parameters, video preprocessor unit 19 can then quantize video data using quantization unit 214. Quantization unit 214 can operate in the same way as described above with reference to Figure 5. After quantization, the video data is now adjusted in the target color space and target color range of the primary target of HDR '216 data. HDR' 216 data can then be be sent to video encoder 20 for compression.
[0164] [0164] Figure 12 is a block diagram illustrating an example of an HDR / WCG reverse conversion apparatus according to the techniques of this disclosure. As shown in Figure 12, the video post processor unit 31 can be configured to apply the reverse of the techniques performed by the video preprocessor unit 19 of Figure 11. In other examples, the techniques of the post processor unit video 31 may be incorporated into, and performed by, the video decoder 30.
[0165] [0165] In one example, the video decoder
[0166] [0166] The video decoder 30 can also be configured to decode and send any one or more syntax elements produced by the DRA parameter estimation unit 212 of Figure 11 to DRA derivation unit parameters 312 of the DRA unit. video post processor 31. The DRA 312 branch unit parameters can be configured to determine the DRA parameters based on one or more syntax elements, as described above. In some examples, the one or more syntax elements explicitly state the DRA parameters. In other examples, the DRA 312 branch unit parameters are configured to derive the DRA parameters using the same techniques used by the DRA 212 parameter estimation unit in Figure 11.
[0167] [0167] The parameters derived by the DRA bypass unit parameters 312 are sent to the reverse adjustment unit 310. The reverse adjustment unit 310 uses the DRA parameters to perform the inverse of the linear DRA adjustment performed by the adjustment unit 210. The reverse adjustment unit 310 can apply the reverse of any of the adjustment techniques described above to adjustment unit 210. In addition, as with the adjustment unit 210, the reverse adjustment unit 310 can apply the reverse DRA before or after any reverse color conversion. As such, the reverse adjustment unit 310 can apply the DRA parameter to the video data in the target color container or the native color container. In some examples, the reverse adjustment unit 310 can be positioned to apply reverse adjustment before the reverse quantization unit 314.
[0168] [0168] The 308 reverse color conversion unit converts the video data from the target color space (for example, YCbCr) to the native color space (for example, RGB). The reverse transfer function 306 then applies an inverse of the transfer function applied by the transfer function 206 to decompress the dynamic range of the video data. In some instances, the resulting video data (RGB target CG 304) is still in the target color range, but is now in the native dynamic range and native color space. Then, the inverse CG converter 302 converts the target RGB RGB 304 into the native color gamut to reconstruct the native CG from RGB 300.
[0169] [0169] In some examples, additional post-processing techniques can be employed by the video post-processor unit 31. The application of the DRA can place the video outside its real native color range. The quantization steps performed by the quantization unit 214 and the inverse quantization unit 314, as well as the up and down sampling techniques performed by the adjustment unit 210 and the reverse adjustment unit 310, can contribute to the color values resulting in the native color container are outside the native color range. When the native color gamut is known (or the lowest primary real content, if indicated, as described above), further processing can be applied to RGB 304 native CG video data to transform color values (for example, RGB or Cb and Cr) back in the desired range as post-processing for DRA. In other examples, this post-processing can be applied after quantization or after applying the DRA.
[0170] [0170] According to one or more examples below, this disclosure describes techniques and devices for adjusting the dynamic range of cross components (CC-DRA) to improve the compression efficiency of image and video encoding systems, such as H.264 / AVC, H.265 / HEVC or next generation of codecs (for example, VVC). More specifically, this describes techniques and devices for deriving CC-DRA parameters that are applied to the chroma component (for example, Cb and Cr), exploring parameters of the DRA function applied to the luma component and / or properties of the processed video signal ( for example, local brightness level, transfer characteristics, color container properties or native color range of the signal).
[0171] [0171] Suppose that a DRA function is applied to the Y component and the CC-DRA function that is applied to the chroma components (for example, Cb, Cr) are specified by the LCS Scb (or Scr) functions. One or more of the following techniques can be applied independently, or in any suitable combination, to derive CC-DRA parameters and their functions from LCS Scb (or Scr).
[0172] [0172] The following examples will be described with reference to the video preprocessor unit 19. However, it should be understood that all processes performed by the video preprocessor unit 19 can also be performed within the video encoder 20 as part of the video encoding loop. The video post-processor unit 31 can be configured to perform the same techniques as the video pre-processor unit 19, but in a reciprocal manner. In particular, the video post processor unit 31 can be configured to determine parameters for CC-DRA of chroma components in the same way as the video preprocessor unit 19. All processes performed by the video post processor unit 31 can also be performed within the video decoder 30 as part of the video decoding loop.
[0173] [0173] As described above with reference to Figure 11 and Figure 12, the video preprocessor unit 19 and the video postprocessor unit 31 can apply DRA to three components of video components of video data (eg example, YCbCr) to obtain a more efficient representation for compression with an image or video compression system (for example, video encoder 20 and video decoder 30). The video preprocessor unit 19 can apply a DRA Sy function to the luma (Y) component and can apply CC-DRA to the chroma components (for example, Cb, Cr) with LCS Scb (or Scr) functions.
[0174] [0174] In an example of the disclosure, the video preprocessor unit 19 can be configured to derive DRA parameters (e.g., scale and offset) for luma components of the video data in the manner described above with reference to Figure 11 The video preprocessor unit 19 can also be configured to derive an offset parameter for chroma components in the manner described above with reference to Figure 11. According to an example of the disclosure, the video preprocessor unit 19 can be configured to derive parameters (for example, scale) of the LCS function Scb or Scr from the parameters of the DRA function applied to component Y, that is, Scb (or Scr) = fun (Sy). That is, the video preprocessor unit 19 can be configured to determine CC-DRA parameters (for example, scale parameters) for a chroma component as a function of derived DRA parameters (for example, scale parameters) for a luma component.
[0175] [0175] In some examples, the video preprocessor unit 19 can be configured to set the LCS function scale factor to chroma values equal to the DRA Sv function scale factor, applied to the chroma component value ( Y):
[0176] [0176] In this example, video preprocessor unit 19 can reuse the derived scale parameter for a luma value (S (Y) y) for use as the scale parameter for the corresponding Cr value (S (Y) cr ) and for use as the scale parameter for the corresponding Cb (S (Y) cb) value. In this context, the corresponding Cr and Cb values are the Cr and Cb values of the same pixel with the luma value.
[0177] [0177] In another example of the disclosure, the video preprocessor unit 19 can be configured to determine luma scale parameters for each of a plurality of codeword value ranges for luminance components of the video data . For example, the total range of codeword values for a luma component can be between 0 and 1023. The video preprocessor unit 19 can be configured to divide the possible codeword values for the luma component into one plurality of ranges and then derive DRA parameters separately for each of the ranges. In some examples, lower scale values can be used for lower intensity luma values (for example, darker values), while higher scale values can be used for higher intensity luma values (for example, brighter values). The total number of luma code words can be divided into any number of tracks, and the tracks can be uniform or non-uniform in size.
[0178] [0178] Consequently, in some examples, the DRA Sy function is defined for a set of bands {Ri}. In this example, the video preprocessor unit 19 can be configured to derive the scale factor of the LCS Scb or Scr functions from the DRA Sy, i scale factor values of the range Ri to which the current Y belongs to: where Y is a Luma value that belongs to the range Ri = [yi, yi + 1- 1]. In other words, the scale factor for a chroma component is a function of the scale factor for the corresponding luma component of the chroma component, depending on the range in which the luma component falls. As such, the video preprocessor unit 19 can be configured to determine chroma scale parameters for chrominance components associated with luminance components that have a first code word range from the plurality of word value ranges code using a function of the luma scale parameter determined for the luminance components with the first range of code word values.
[0179] [0179] However, the chroma scale values can also be derived in ranges of chroma code words, and the range of code words values for chroma components for which the chroma scale parameters are derived may not overlap directly to the code word value ranges for luma components for which luma scale parameters are derived. For example, a single range of chroma codeword values can overlap with several range of luma codeword values. In this example, the fun derivation function (S (Y) y, i) to derive scale values from chroma components can be implemented as a zero order approximation (for example, an average of all scale values for the Ri range of luma scale values) or as a higher order approximation (for example, interpolation, curve fitting, low-pass filtering, etc.).
[0180] [0180] In some examples, the video preprocessor unit 19 can be configured to determine chroma scale parameters for LCS Scb or Scr functions as a function of various LRA scale factors through a general process of curve fit:
[0181] [0181] Consequently, the video preprocessor unit 19 can be configured to determine chroma scale parameters for chrominance components associated with luminance components having a first code word range and at least a second value range code word from the plurality of code word value ranges using a function of the luma scale parameter determined for the luminance components having the first code word value range and the second code word value range.
[0182] [0182] In some examples, the luma scale parameters are constant within a specific range of the luma code word values. As such, the luma scale parameters can be seen as a discontinuous function (for example, step by step) over the entire range of luma code words. Instead of using the luma scale discontinuous parameters to derive the chroma scale parameters, the video preprocessor unit 19 can be configured to apply a linearization process to the discontinuous function to produce linearized luma scale parameters. The video preprocessor unit 19 can be further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters. In some examples, the linearization process is one or more of a linear interpolation process, a curve fitting process, an averaging process, or a higher order approximation process.
[0183] [0183] Regardless of how the luma scale parameters and chroma scale parameters are derived, the video preprocessor unit 19 can be configured to perform a dynamic range adjustment process on the luminance components using the scale parameters luma and perform a dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters, as described above with reference to Figure 11.
[0184] [0184] In another example of the disclosure, the video preprocessor unit 19 can be configured to derive parameters (for example, scale parameters) from the LCS Scb or Scr function from the DRA function parameters applied to component Y Sy and a quantization parameter (QP) of the codec (for example, video encoder 20), such as quantization parameter offset (QP), deltaQP, chromaQP, or other parameters that specify granularity of the codec quantization. A deltaQP can indicate the difference between the QP for one block and the QP for another block. A chromaQP offset can be added to the QP of a luma component to determine the QP of a chroma component.
[0185] [0185] Consequently, the video preprocessor unit 19 can be configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and a quantization parameter used to decode the chrominance components.
[0186] [0186] Below is an example:
[0187] [0187] In some examples, the correspondence between the coding QP for a current component and an adjustment parameter for an S cb or S cr function can be tabulated and used to adjust the approximation fun function.
[0188] [0188] In some examples, a set of functions {fun (S y, QPCr, id} can be provided for video decoder 30 and / or video post processor unit 31 as side information and a selection between these functions can be made by a derivation of the bitstream function identification (id) or from the decoder side analysis of the encoding parameters, such as QP, encoding modes, encoded image type (I, B, P), availability of the prediction or encoding mode.
[0189] [0189] In another example of the disclosure, the video preprocessor unit 19 can be configured to derive DRA parameters (for example, scale parameters) from the LCS Scb or Scr function from the DRA function parameters applied to the Y Sy component and colorimetry parameters of processed video data and the container used (for example, the primary colors of the native range and the color container used). The colorimetry parameters of the processed video data (color representation parameter) and the container used can be multiplication factors a and f1 which can be the same for each of the chroma components (Cr and Cb) or can be independent for each one of the chroma components.
[0190] [0190] In some examples, parameters a and f1 can be defined by a single multiplier parameter, for example, a = 1.0698 and f1 = 2.1735 for BT.709 video content represented in a BT color container. 2020.
[0191] [0191] In some examples, parameters a and f1 can be transmitted in an encoded bit stream. In other examples, parameters a and f1 can be derived either by the video preprocessor unit 19 / video encoder 20 or by the video post processor unit 31 / video decoder 30. Also, the preprocessor unit Video 19 can be configured to determine parameters a and f1 in various ranges of code words for chroma components. That is, parameters a and f1 can be a single value for all chroma components or a set of values (for example, for chroma component code word ranges).
[0192] [0192] In other examples, the video preprocessor unit 19 can be configured to derive parameter a and f1 from colorimetry parameters (for example, primary colors and white point coordinates of the color container and native color gamut).
[0193] [0193] In other examples, the video preprocessor unit 19 can be configured to determine parameter a and f1 as a function of a luma Y value and / or chroma sample value.
[0194] [0194] In other examples, the video preprocessor unit 19 can be configured to derive, and / or tabulate and access the value of parameters a and f1 using other characteristics of the color space, such as non-linearity (for example, function transform) or color space used. Below is an example:
[0195] [0195] Examples of transfer characteristics can be transfer functions of BT.709, BT.2100 and / or (PQ) /BT.2100 (EILG). The video preprocessor unit 19 can be configured to signal an id syntax element of the transform characteristic whose value specifies the non-linearity used (e.g., transform function).
[0196] [0196] In some examples, the colorimetry parameters used for LCS derivation can be derived from the transmitted bit stream or can be derived from sample values of the video content on the decoder side. In some examples, parameters a and f1 or a type of derivation function can be signaled via the bit stream or made available to the decoder as side information and identified with an id parameter signaled in the bit stream.
[0197] [0197] As described above, each of the above techniques can be used in any combination. For example, in one implementation, the video preprocessor unit 19 can be configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters, the quantization parameter used for decode the chrominance components, and a color representation parameter derived from the characteristics of the chroma component of the video data. In this example, the video preprocessor unit 19 can also be configured to linearize the luma scale parameters.
[0198] [0198] In another example of the disclosure, the video preprocessor unit 19 can be configured to derive parameters (for example, scale parameters) from the LCS Scb or Scr function as an overlay of the DRA function applied to the component Y Sy and DRA functions specific to current color components, as defined in Equations (3) and (4). The scale parameters derived from Equations (3) and (4) can be considered as initial chroma scale parameters:
[0199] [0199] Below is an example:
[0200] [0200] As such, in an example of the disclosure, the video preprocessor unit 19 can be configured to determine initial chroma scale parameters for the chrominance components of the video data, and determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and the initial chroma scale parameters.
[0201] [0201] Below are descriptions of several non-limiting examples of implementing the techniques described above.
[0202] [0202] Suppose a Y component is processed with DRA defined by a piecewise linear function Sy, i (Y) as a set of bands without overlap {Ri}, where Y is a processed luma sample and i is a partitioning id I laugh. The sub-ranges of the DRA Ri function are defined by the minimum and maximum values belonging to the range, for example, [yi, ... yi + 1-1], where i denotes the index of the ranges that consist of DRA, yi term indicates the initial value for hi where N is the total number of sub-bands used. A scale value Sy, i and a displacement value 0y, i are defined for each Ri independently and applied to all samples Y with value that belongs to the subrange [yi, yi + 1-1].
[0203] [0203] In some examples, an inverse DRA function can be implemented on the decoder side, as shown in Equation (2) and Figure 12 above. Figure 13 shows an example of a DRA 183 mapping function applied to the input luma component Y (input code words) and a “No DRA” function 181, when no DRA is conducted.
[0204] [0204] In some examples, the LCS Scb or Scr function for chroma scale factors can be defined by setting the LCS function value equal to the DRA Sy function scale factor applied to the Y value, as shown in Equation (22 ).
[0205] [0205] Since the DRA Sy function is defined with an equal scale for all Y samples of sub-range i, the derived LCS has a discontinuous function (for example, step function), shown in graph 191 of Figure 14. In some examples, the video preprocessor unit 19 can derive the functions from LCS Scb or Scr with a linearization of the scale factors used for the DRA function of the luma (shown as curve 193 in Figure 14).
[0206] [0206] In some examples, the linearization fun (Sy (Y)) can be conducted by local approximation of the scale parameters on the neighboring partitions i and i + 1, as follows. For i = 0..N-2, derive the size of neighboring sub-bands:
[0207] [0207] Define the Rci approximation sub-bands for the LCS function as a half offset range for the Ri sub-bands of the DRA function:
[0208] [0208] The video preprocessor unit 19 can derive the LCS scale factors for Y = Yci ... Yc + 1-1 as a linear interpolation of the DRA scale factors of neighboring sub-bands:
[0209] [0209] Again, an example of a linearized LCS function is shown as lot 193 in Figure 14.
[0210] [0210] In another example, the video preprocessor unit 19 can derive LCS Scb or Scr function parameters from DRA function parameters applied to the Y Scy component and codec quantization parameters, such as QP, deltaQP and / or chromaQP parameter.
[0211] [0211] An adjustment to the LCS parameters {1cs_param1, 1cs_param2, 1cs_param3} proportional to the quantization parameters (eg QP) used to encode the current image / slice or transform block can be used to approximate the 1CS of the scale function of DRA luma Sy, and removing its discontinuous function nature.
[0212] [0212] For example, the interpolation of the chroma scale function for the range [Yci ... Yci + 1], which is shown in Equation (33), can be performed through a sigmoid function being adjusted as a function of QP . An example is shown below.
[0213] [0213] For the values Y = Yci ... Yc + 1-1, a sigmoid function can be used for interpolation between scales of neighborhood bands.
[0214] [0214] In yet another example, a discontinuous adaptive function or its approximations can be used for interpolation.
[0215] [0215] With this method, the controlled smoothness can be introduced in the discontinuous function of the 1CS function in the case of differences between scales of the neighboring sub-bands. A set of smoothing functions can be provided next to the decoder (for example, video decoder 30 and / or video post processor unit 31) as side information, and the selection of the function ID can be signaled in the stream. bits. Alternatively, a smoothing function can be signaled in the bit stream.
[0216] [0216] The derivation of the smoothing function can also be tabulated in the form of a relationship between a QP and the parameters of a smoothing function or function ID, as shown below. Role ID
[0217] [0217] Figure 15 shows an example of using a set of sigmoid functions to provide a controlled smoothness of the discontinuous function to Scb (or Cr) at the edge of two neighboring sub-bands.
[0218] [0218] An example of an application-controlled smoothness function for the derivation of the 1CS_ function with Equation (33) is shown as graph 195 (smoothed DRA scales) in Figure 16. Graph 197 shows the original DRA scales and Graph 199 shows the linearized DRA scales. Graph 195 (smoothed DRA curve)
[0219] [0219] In some examples, the function of 1CS_Scb (or Scr) can be further adapted to the properties of colorimetry, as shown in Equation (26).
[0220] [0220] Set primeT target container properties and primeN native color gamut using primary color coordinates: primeT = (xRc, yRc; xGc, yGc; xBc, yBc), primeN = (xRn, yRn; xGn, yGn ; xBn, yBn) and the white point coordinate whiteP = (xW, yW). Examples for some colors are provided in Table 1. With that,
[0221] [0221] In another example, the value of the alpha and beta multipliers can be derived from QP-type parameters that adjust the codec quantization scheme for an encoded video color container: where deltaCrQP / deltaCbQP is an adjustment in the QP settings of the codec to meet color container properties.
[0222] [0222] In some examples, the parameters of the techniques described above can be estimated on the encoder side (for example, video encoder 20 and / or video preprocessor unit 19) and signaled through a bit stream (metadata) , SEI, VUI, or SPS / PPS message or slice header, etc.). The video decoder 30 and / or video post-processor unit 31 receives parameters from a bit stream.
[0223] [0223] In some examples, the parameters of the techniques described above are derived both in the video encoder 20 / video preprocessor unit 19 and in the video decoder 30 / video post processor unit 31 through a specific process from the input signal or other available parameters associated with the input signal and processing flow.
[0224] [0224] In some examples, they are explicitly flagged and are sufficient to run the DRA on the video decoder 30 / video post processor unit 31. In still other examples, the parameters of the techniques described above are derived from other signal parameters input parameters, for example, parameters of the input color range and the target color container (primary colors).
[0225] [0225] Figure 17 is a block diagram illustrating an example of video encoder 20 that can implement the techniques of this disclosure. As described above, DRA techniques can be performed by the video preprocessor unit 19 outside the video encoder encoding loop 20 or inside the video encoder encoding loop 20 (for example, before prediction). The video encoder 20 can perform intra- and inter-encoding of video blocks within video slices in a target color container that has been processed by the video preprocessor unit 19. Intra-coding depends on the spatial prediction for reduce or remove the temporal redundancy of the video in adjacent frames or images in a video sequence. Intra mode (I mode) can refer to any of several space-based encoding modes. Among modes, such as unidirectional prediction (P mode) or biprediction (B mode), they can refer to any of several time-based encoding modes.
[0226] [0226] As shown in Figure 17, video encoder 20 receives a current video block within a video frame to be encoded. In the example in Figure 17, video encoder 20 includes mode selection unit 40, a video data memory 41, decoded image buffer 64, adder 50, transform processing unit 52, quantization unit 54, and unit entropy coding unit 56. The mode selection unit 40, in turn, includes motion compensation unit 44, motion estimate unit 42, intra-prediction processing unit 46, and partition unit 48. For reconstruction video block, video encoder 20 also includes reverse quantization unit 58, reverse transform processing unit 60, and adder 62. An unlock filter (not shown in Figure 17) can also be included to filter the block boundaries to remove blocking artifacts from the reconstructed video. If desired, the unlock filter would normally filter the output of adder 62. Additional filters (loop or post-loop) can also be used in addition to the unlock filter. These filters are not shown for the sake of brevity, but, if desired, they can filter the output of the adder 50 (like a loop filter).
[0227] [0227] The video data memory 41 can store video data to be encoded by the components of the video encoder 20. The video data stored in the video data memory 41 can be obtained, for example, from the video source. video 18. The decoded image buffer 64 can be a reference image memory that stores reference video data for use in encoding video data by video encoder 20, for example, in intra or inter-encoding modes. The video data memory 41 and the decoded image buffer 64 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM) ), Resistive RAM (RRAM), or other types of memory devices. The video data memory 41 and the decoded image buffer 64 can be provided by the same memory device or by separate memory devices. In several examples, the video data memory 41 may be on the chip with other components of the video encoder 20 or off the chip with respect to those components.
[0228] [0228] During the encoding process, video encoder 20 receives a frame or video slice to be encoded. The frame or slice can be divided into several blocks of video. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive encoding of the received video block in relation to one or more blocks in one or more reference frames to provide time prediction. The intra-prediction processing unit 46 may alternatively perform intra-predictive encoding of the received video block with respect to one or more neighboring blocks in the same frame or slice as the block to be encoded to provide spatial prediction. The video encoder 20 can perform several encoding passes, for example, to select an appropriate encoding mode for each block of video data.
[0229] [0229] In addition, partition unit 48 can partition blocks of video data into sub-blocks, based on the evaluation of previous partitioning schemes in previous encoding passages. For example, partition unit 48 can initially partition a frame or slice into LCUs and partition each of the LCUs into sub-UCs based on rate distortion analysis (for example, rate distortion optimization). The mode selection unit 40 can also produce a quaternary tree and / or QTBT data structure indicative of partitioning an LCU into sub-UCs. Quaternary tree leaf node CUs can include one or more PUs and one or more TUs.
[0230] [0230] The mode selection unit 40 can select one of the coding modes, intra or inter, for example, based on error results, and provides the resulting intra or inter-coded block to the adder 50 to generate data from residual blocks and for adder 62 to reconstruct the coded block for use as a frame of reference. The mode selection unit 40 also provides syntax elements, such as motion vectors, intra-mode indicators, partition information and other syntax information, for the entropy coding unit 56.
[0231] [0231] The motion estimation unit 42 and the motion compensation unit 44 can be highly integrated, but are illustrated separately for conceptual purposes. Motion estimation, performed by motion estimation unit 42, is the process of generating motion vectors, which estimate motion for video blocks. A motion vector, for example, may indicate the displacement of a PU from a video block within a current frame or video image in relation to a predictive block within a reference image (or other encoded unit) in relation to the current block being encoded within the current image (or another encoded unit). A predictive block corresponds to the one that approaches the code to be coded, in terms of pixel difference, which can be determined by the sum of the absolute difference (SAD), sum of the square difference (SSD) or other difference metrics. In some examples, the video encoder 20 can calculate values for sub-integer pixel positions of reference images stored in the decoded image buffer 64. For example, the video encoder 20 can interpolate quarter pixel position values, eighth pixel positions or other fractional pixel positions of the reference image. Therefore, the motion estimation unit 42 can perform a motion search against complete pixel positions and fractional pixel positions and generate a motion vector with fractional pixel precision.
[0232] [0232] The motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-encoded slice by comparing the position of the PU with the position of a predictive block of a reference image. The reference image can be selected from a first reference image list (List 0) or from a second reference image list (List 1), each of which identifies one or more reference images stored in the decoded image buffer 64. The motion estimation unit 42 sends the calculated motion vector value to the entropy coding unit 56 and motion compensation unit 44.
[0233] [0233] Motion compensation, performed by motion compensation unit 44, may involve fetching or generating the predictive block based on the motion vector determined by motion estimation unit 42. Again, motion estimation unit 42 and the motion compensation unit 44 can be functionally integrated, in some examples. Upon receiving the motion vector for the PU of the current video block, the motion compensation unit 44 can locate the predictive block to which the motion vector points in one of the reference image lists. Adder 50 forms a residual video block by subtracting the pixel values from the predictive block from the pixel values of the current video block being encoded,
[0234] [0234] The intra-prediction processing unit 46 can intra-predict a current block, as an alternative to the inter-prediction performed by the motion estimation unit 42 and motion compensation unit 44, as described above. In particular, the intra-prediction processing unit 46 can determine an intra-prediction mode for use in encoding a current block. In some instances, the intra-prediction processing unit 46 may encode a current block using various intra-prediction modes, for example, during separate coding passes, and intra-prediction processing unit 46 (or selection unit so 40, in some examples) you can select a suitable intra-prediction mode for use from the tested modes.
[0235] [0235] For example, intra-prediction processing unit 46 can calculate rate distortion values using rate distortion analysis for the various test prediction modes tested and select the intra prediction mode with the best rate distortion between the tested modes. Rate distortion analysis generally determines an amount of distortion (or error) between an encoded block and an original uncoded block that was encoded to produce the encoded block, as well as a bit rate (that is, a number of bits) used to produce the coded block. The intra-prediction processing unit 46 can calculate the ratios from the distortions and rates for the various coded blocks to determine which intra-prediction mode exhibits the best rate distortion value for the block.
[0236] [0236] After selecting an intra-prediction mode for a block, the intra-prediction processing unit 46 can provide information indicative of the intra-prediction mode selected for the block for the entropy coding unit 56. The entropy coding 56 can encode information indicating the selected intra-prediction mode. The video encoder 20 may include in the transmitted bit stream configuration data, which may include a plurality of index tables in the intra-prediction mode and a plurality of modified index tables in the intra-prediction mode (also known as codeword mapping tables), multi-block coding context definitions and indications of a most likely intra-prediction mode, an intra-prediction mode index table and a modified intra-prediction mode index table for use in each of the contexts.
[0237] [0237] Video encoder 20 forms a residual video block by subtracting the prediction data from mode selection unit 40 from the original video block being encoded. Adder 50 represents the component or components that perform this subtraction operation. The transform processing unit 52 applies a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform, to the residual block, producing a video block comprising residual transform coefficient values. The transform processing unit 52 can carry out other transforms that are conceptually similar to the DCT. Wavelet transforms, whole transforms, subband transforms or other types of transformations can also be used. In any case, the transform processing unit 52 applies the transform to the residual block, producing blocks of residual transform coefficients. The transform can convert the residual information from a pixel value domain to a transform domain, such as a frequency domain. The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54.
[0238] [0238] The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 can then scan the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.
[0239] [0239] After quantization, the entropy coding unit 56 encodes the entropy of the quantized transform coefficients. For example, entropy coding unit 56 can perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic (CABAC), context-adaptive binary arithmetic (SBAC) based on syntax, entropy encoding of probability range partitioning (PIPE) or other entropy coding technique. In the case of context-based entropy coding, the context can be based on neighboring blocks. After entropy encoding by entropy encoding unit 56, the encoded bit stream can be transmitted to another device (e.g., video decoder 30) or archived for later transmission or retrieval.
[0240] [0240] The inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain, for example, for later use as a reference block. The motion compensation unit 44 can calculate a reference block by adding the residual block to a predictive block from one of the decoded image buffer frames 64. The motion compensation unit 44 can also apply one or more interpolation filters for the reconstructed residual block to calculate sub-integer pixel values for use in motion estimation. Adder 62 adds the reconstructed residual block to the motion compensation prediction block produced by the motion compensation unit 44 to produce a reconstructed video block for storage in decoded image buffer 64. The reconstructed video block can be used by the unit motion estimation 42 and motion compensation unit 44 as a reference block for inter-encoding a block in a subsequent video frame.
[0241] [0241] Figure 18 is a block diagram illustrating an example of video decoder 30 that can implement the techniques of this disclosure. As described above, reverse DRA techniques can be performed by the video post-processor unit 31 outside the video decoder decode loop 30 or within the video decoder decode loop 30 (for example, after filtering and before decoded image buffer 82). In the example in Figure 18, the video decoder 30 includes an entropy decoding unit 70, a video data memory 71, motion compensation unit 72, intra-prediction processing unit 74, inverse quantization unit 76, reverse transform processing unit 78, decoded image buffer 82 and adder 80. The video decoder 30 may, in some instances, perform a generally reciprocal decode pass) for the encoding pass described in relation to the video encoder 20 ( Figure 17). The motion compensation unit 72 can generate prediction data based on motion vectors received from the entropy decoding unit 70, while the intra prediction processing unit 74 can generate prediction data based on intra mode indicators. -prediction received from entropy decoding unit 70.
[0242] [0242] The video data memory 71 can store video data, as an encoded video bit stream, to be decoded by the video decoder components 30. The video data stored in the video data memory 71 can be obtained, for example, from a computer-readable medium 16, for example, from a local video source, such as a camera, through wired or wireless network communication of video data or by accessing physical data storage media. The video data memory 71 can form an encoded image buffer (CPB) that stores encoded video data from an encoded video bit stream. The decoded image buffer 82 can be a reference image memory that stores reference video data for use in decoding video data by the video decoder 30, for example, in intra or inter-encoding modes. Video data memory 71 and decoded image buffer 82 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM) ), Resistive RAM (RRAM), or other types of memory devices. The video data memory 71 and the decoded image buffer 82 can be provided by the same memory device or separate memory devices. In several examples, the video data memory 71 may be on the chip with other components of the video decoder 30 or off the chip with respect to those components.
[0243] [0243] During the decoding process, the video decoder 30 receives an encoded video bit stream that represents video blocks from an encoded video slice and associated syntax elements from the video encoder 20. The entropy decoding unit 70 of the entropy video decoder 30 decodes the bit stream to generate quantized coefficients, motion vectors or intra-prediction mode indicators and other syntax elements. The entropy decoding unit 70 forwards the motion vectors to and other syntax elements to the motion compensation unit 72. The video decoder 30 can receive the syntax elements at the video slice level and / or at the video level. video unit.
[0244] [0244] When the video slice is encoded as an intra-encoded slice (I), the intra prediction processing unit 74 can generate prediction data for a video block of the current video slice based on a prediction mode signaled and previously decoded block data from the current frame or image. When the video frame is encoded as an inter-encoded slice (i.e., B or P), the motion compensation unit 72 produces predictive blocks for a video block of the current video slice based on the motion vectors and others syntax elements received from the entropy decoding unit 70. Predictive blocks can be produced from one of the reference images within one of the reference image lists. The video decoder 30 can build the reference image lists, List 0 and List 1, using standard construction techniques based on reference images stored in the decoded image buffer 82. The motion compensation unit 72 determines prediction information for a video block from the current video slice analyzing the motion vectors and other syntax elements and uses the prediction information to produce the predictive blocks for the current video block being decoded. For example, the motion compensation unit 72 uses some of the syntax elements received to determine a prediction mode (for example, intra or inter-prediction) used to encode the video blocks of the video slice, a type of slice between predictions (for example, slice B or slice P), construction information for one or more lists of reference images for the slice, motion vectors for each encoded video block of the slice, inter-prediction status for each video block encoded slice and other information to decode the video blocks in the current video slice.
[0245] [0245] The motion compensation unit 72 can also perform interpolation based on interpolation filters. The motion compensation unit 72 can use interpolation filters as used by the video encoder 20 during the encoding of the video blocks to calculate interpolated values for sub-integer pixels of reference blocks. In this case, the motion compensation unit 72 can determine the interpolation filters used by the video encoder 20 from the received syntax elements and use the interpolation filters to produce predictive blocks.
[0246] [0246] The inverse quantization unit 76 quantizes inversely, that is, decantifies, the quantized transform coefficients provided in the bit stream and decoded by the entropy decoding unit
[0247] [0247] After the motion compensation unit 72 generates the predictive block for the current video block based on the motion vectors and other syntax elements, video decoder 30 forms a decoded video block summing up the residual blocks of the reverse transform processing unit 78 with the corresponding predictive blocks generated by the motion compensation unit 72. Adder 80 represents the component or components that perform this sum operation. If desired, an unlock filter can also be applied to filter the decoded blocks in order to remove blocking artifacts. Other loop filters (in the encoding loop or after the encoding loop) can also be used to smooth out pixel transitions, or otherwise improve video quality. The video blocks decoded in a given frame or image are then stored in a decoded image buffer 82, which stores reference images used for subsequent motion compensation. The decoded image buffer 82 also stores decoded video for later presentation on a display device, such as display device 32 in Figure 1.
[0248] [0248] Figure 19 is a flow chart showing an example of the video processing technique of the disclosure. The techniques of Figure 19 can be performed by the video preprocessor unit 19 and / or video encoder 20, as described above. In an example of the disclosure, the video preprocessor unit 19 and / or video encoder 20 can be configured to receive video data (1900), determine luma scale parameters for each of a plurality of value ranges. codeword for luminance components of the video data (1902), and perform a dynamic range adjustment process on the luminance components using the luma scale parameters (1904). The video preprocessor unit 19 and / or video encoder 20 can be further configured to determine chroma scale parameters for chrominance components of the video data using a function of the luma scale parameters (1906), and perform a process of dynamic range adjustment in the chrominance components of the video data using the chroma scale parameters (1908). The video encoder 20 can then be configured to encode the video data (1910).
[0249] [0249] In one example, determining the chroma scale parameters, video preprocessor unit 19 and / or video encoder 20 are further configured to determine chroma scale parameters for chrominance components associated with luminance components having a first codeword value range of the plurality of codeword value ranges using a function of the luma scale parameters determined for the luminance components having the first codeword value range.
[0250] [0250] In another example, determining the chroma scale parameters, video preprocessor unit 19 and / or video encoder 20 are further configured to determine chroma scale parameters for chrominance components associated with luminance components having a first range of code word values and a second range of code word values from the plurality of range of code word values using a function of the luma scale parameters determined for the luminance components having the first range of word values -code and the second range of code word values.
[0251] [0251] In another example, the luma scaling parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, and video preprocessor unit 19 and / or video encoder 20 are further configured to apply a linearization process to the discontinuous function to produce linearized luma scale parameters, and determine the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters. In one example, the linearization process is one or more of a linear interpolation process, a curve fitting process, an averaging process, or a higher order approximation process.
[0252] [0252] In another example, determining the chroma scale parameters, video preprocessor unit 19 and / or video encoder 20 are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and a quantization parameter used to decode the chrominance components.
[0253] [0253] In another example, determining the chroma scale parameters, video preprocessor unit 19 and / or video encoder 20 are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters, the quantization parameter used to decode the chrominance components, and a color representation parameter derived from the characteristics of the chroma component of the video data. In one example, the color representation includes a transfer function associated with the color / video data container.
[0254] [0254] In another example, the video preprocessor unit 19 and / or video encoder 20 are further configured to determine initial chroma scale parameters for the chrominance components of the video data, in which to determine the parameters of chroma scale, video preprocessor unit 19 and / or video encoder 20 are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and the scale parameters chroma initials.
[0255] [0255] In another example, the video preprocessor unit 19 and / or video encoder 20 determine luma shift parameters for the luminance components, perform the dynamic range adjustment process on the luminance components using the parameters luma scale parameters and luma displacement parameters, determine chroma displacement parameters for the chrominance components, and perform the dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters and chroma displacement parameters .
[0256] [0256] Figure 20 is a flow chart showing another example of the video processing technique of the disclosure. The techniques of Figure 20 can be performed by the video post-processor unit 31 and / or video decoder 30, as described above. In an example of the disclosure, video decoder 30 can be configured to decode video data (2000) and video post processor unit 31 and / or video decoder 30 can be configured to receive video data (2002) . The video post-processor unit 31 and / or video decoder 30 can be further configured to determine luma scaling parameters for each of a plurality of codeword value ranges for luminance components of the video data (2004 ), and perform an inverse dynamic range adjustment process on the luminance components using the luma scale parameters (2006). The video post-processor unit 31 and / or video decoder 30 can be further configured to determine chroma scale parameters for chrominance components of the video data using a function of the luma scale parameters (2008), and perform a process of inverse dynamic range adjustment in the chrominance components of the video data using the chroma scale parameters (2010).
[0257] [0257] In one example, determining the chroma scale parameters, video post processor unit 31 and / or video decoder 30 are further configured to determine chroma scale parameters for chrominance components associated with luminance components having a first codeword value range of the plurality of codeword value ranges using a function of the luma scale parameters determined for the luminance components having the first codeword value range.
[0258] [0258] In another example, determining the chroma scale parameters, video post processor unit 31 and / or video decoder 30 are further configured to determine chroma scale parameters for chrominance components associated with luminance components having a first range of code word values and a second range of code word values from the plurality of range of code word values using a function of the luma scale parameters determined for the luminance components having the first range of word values -code and the second range of code word values.
[0259] [0259] In another example, the luma scaling parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, and video post-processor unit 31 and / or video decoder 30 are further configured to apply a linearization process to the discontinuous function to produce linearized luma scale parameters, and determine the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters. In one example, the linearization process is one or more of a linear interpolation process, a curve fitting process, an averaging process, or a higher order approximation process.
[0260] [0260] In another example, determining the chroma scale parameters, video post processor unit 31 and / or video decoder 30 are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and a quantization parameter used to decode the chrominance components.
[0261] [0261] In another example, determining the chroma scale parameters, video post processor unit 31 and / or video decoder 30 are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters, the quantization parameter used to decode the chrominance components, and a color representation parameter derived from the characteristics of the chroma component of the video data. In one example, the color representation parameter includes a transfer function associated with the color / video data container.
[0262] [0262] In another example, the video post processor unit 31 and / or video decoder 30 are further configured to determine initial chroma scale parameters for the chrominance components of the video data, in which to determine the parameters of chroma scale,
[0263] [0263] In another example, the video post processor unit 31 and / or the video decoder 30 determine luma shift parameters for the luminance components, perform the process of adjusting the inverse dynamic range on the luminance components using the luma scale parameters and the luma displacement parameters, determine chroma displacement parameters for the chrominance components, and perform the inverse dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters and parameters chroma shift.
[0264] [0264] Certain aspects of this disclosure have been described with respect to HEVC, extensions of the HEVC, JEM and VVC standards for illustration purposes. However, the techniques described in this disclosure may be useful for other video encoding processes, including other standard or proprietary video encoding processes under development or not yet developed, including VVC.
[0265] [0265] A video encoder, as described in this disclosure, can refer to a video encoder or video decoder. Likewise, a video encoding unit can refer to a video encoder or decoder. Likewise, video encoding can refer to video encoding or decoding, as applicable.
[0266] [0266] It must be recognized that, depending on the example, certain acts or events of any of the techniques described here can be performed in a different sequence, can be added, merged or deleted entirely (for example, not all the acts described or events are necessary to practice the techniques). In addition, in certain examples, acts or events can be performed simultaneously, for example, through multithreaded processing, interrupt processing or multiple processors, instead of sequentially.
[0267] [0267] In one or more examples, the functions described can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, functions can be stored or transmitted as one or more instructions or codes on computer-readable media and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to a tangible medium, such as data storage media or communication media, including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, computer-readable media can generally correspond to (1) tangible computer-readable storage media that is not transient or (2) a communication medium such as a signal or carrier wave.
[0268] [0268] By way of example, and without limitation, these computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other means that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly called a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave , coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies, such as infrared, radio and microwave, are included in the media definition. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other transient media, but are directed at tangible and non-transitory storage media. Floppy disk and disk, as used here, include compact disk (CD),
[0269] [0269] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs) or other equivalent integrated or discrete logic circuit. Therefore, the term "processor", as used herein, can refer to any of the previous structures or any other structure suitable for the implementation of the techniques described here. In addition, in some respects, the functionality described here can be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, the techniques can be fully implemented in one or more circuits or logic elements.
[0270] [0270] The techniques of this disclosure can be implemented in a wide variety of devices or devices, including a cell phone, an integrated circuit (IC) or a set of ICs (for example, a chipset). Various components, modules or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Instead, as described above, multiple units can be combined into one codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with appropriate software and / or firmware.
[0271] [0271] Several examples have been described. These and other examples are within the scope of the following claims.

权利要求:
Claims (30)
[1]
1. Video data processing method, the method comprising: receiving video data; determining luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data; perform a dynamic range adjustment process on the luminance components using the luma scale parameters; determine chroma scale parameters for chrominance components of video data using a function of the luma scale parameters; and perform a dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters.
[2]
A method according to claim 1, wherein determining the chroma scale parameters comprises: determining chroma scale parameters for chrominance components associated with luminance components having a first range of code word values from the plurality of ranges of codeword values using a function of the luma scaling parameters determined for the luminance components having the first range of codeword values.
[3]
A method according to claim 1, wherein determining the chroma scale parameters comprises: determining chroma scale parameters for chrominance components associated with luminance components having a first range of keyword values
code and a second code word range from the plurality of code word value ranges using a luma scale parameter function determined for the luminance components having the first code word value range and the second code word values.
[4]
Method according to claim 1, in which the luma scale parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, the method further comprising: applying a linearization process to the discontinuous function to produce linearized luma scale parameters; and determining the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters.
[5]
5. Method according to claim 4, wherein the linearization process is one or more of a linear interpolation process, a curve fitting process, an averaging process, a low-pass filtering process, or a higher order approach process.
[6]
6. The method of claim 1, wherein determining the chroma scale parameters further comprises: determining the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and a quantization parameter used to decode the chrominance components.
[7]
7. The method of claim 6, wherein determining the chroma scale parameters further comprises: determining the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters, the quantization parameter used to decode the chrominance components, and a color representation parameter derived from the chroma component characteristics of the video data.
[8]
A method according to claim 7, wherein the color representation parameter includes a transfer function associated with the video data.
[9]
9. Method according to claim 1, further comprising: determining initial chroma scale parameters for the chrominance components of the video data, wherein determining the chroma scale parameters comprises determining the chroma scale parameters for the chrominance components video data using a function of the luma scale parameters and the initial chroma scale parameters.
[10]
A method according to claim 1, further comprising: determining luma displacement parameters for the luminance components; perform the dynamic range adjustment process on the luminance components using the luma scale parameters and the luma displacement parameters; determine chroma shift parameters for the chrominance components; and perform the dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters and the chroma shift parameters.
[11]
11. Method, according to claim 1, further comprising: encoding the video data after performing the dynamic range adjustment process on the luminance components and after performing the dynamic range adjustment process on the chrominance components.
[12]
12. Method according to claim 1, wherein the dynamic range adjustment process is a reverse dynamic range adjustment process, the method further comprising: decoding the video data before performing the dynamic range adjustment process reverse on the luminance components and before performing the inverse dynamic range adjustment process on the chrominance components.
[13]
13. Apparatus configured to process video data, the apparatus comprising: a memory configured to store video data; and one or more processors in communication with the memory, the one or more processors configured to: receive the video data; determining luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data;
perform a dynamic range adjustment process on the luminance components using the luma scale parameters; determine chroma scale parameters for chrominance components of video data using a function of the luma scale parameters; and perform a dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters.
[14]
Apparatus according to claim 13, wherein determining the chroma scale parameters, the one or more processors are further configured to: determine chroma scale parameters for chrominance components associated with luminance components having a first range of values code word of the plurality of code word value ranges using a function of the luma scale parameters determined for the luminance components having the first code word value range.
[15]
Apparatus according to claim 13, wherein determining the chroma scale parameters, the one or more processors are further configured to: determine chroma scale parameters for chrominance components associated with luminance components having a first range of values code word and a second code word range from the plurality of code word value ranges using a function of the luma scale parameters determined for the luminance components having the first code word value range and the second range of code word values.
[16]
Apparatus according to claim 13, in which the luma scale parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, in which the one or more processors are further configured to: apply a linearization process to the discontinuous function to produce linearized luma scale parameters; and determining the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters.
[17]
Apparatus according to claim 16, wherein the linearization process is one or more of a linear interpolation process, a curve fitting process, an averaging process, a low-pass filtering process, or a higher order approach process.
[18]
18. Apparatus according to claim 13, wherein determining the chroma scale parameters, the one or more processors are further configured to: determine the chroma scale parameters for the chrominance components of the video data using a function of the parameters luma scale and a quantization parameter used to decode the chrominance components.
[19]
19. Apparatus according to claim 18, in which to determine the additional chroma scale parameters, the one or more processors are further configured to:
determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters, the quantization parameter used to decode the chrominance components, and a color representation parameter derived from the component characteristics chroma of the video data.
[20]
Apparatus according to claim 19, wherein the color representation parameter includes a transfer function associated with the video data.
[21]
21. Apparatus according to claim 13, wherein the one or more processors are further configured to: determine initial chroma scale parameters for the chrominance components of the video data, in which determine the chroma scale parameters, the one or more processors are further configured to determine the chroma scale parameters for the chrominance components of the video data using a function of the luma scale parameters and the initial chroma scale parameters.
[22]
22. Apparatus according to claim 13, wherein the one or more processors are further configured to: determine luma displacement parameters for the luminance components, perform the dynamic range adjustment process on the luminance components using the luminance parameters luma scale and luma displacement parameters; determine chroma shift parameters for the chrominance components, and perform the dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters and the chroma shift parameters.
[23]
23. Apparatus according to claim 13, in which the one or more processors are further configured to: encode the video data after performing the dynamic range adjustment process on the luminance components and after performing the range adjustment process dynamics in the chrominance components.
[24]
24. Apparatus according to claim 23, further comprising: a camera configured to capture video data.
[25]
25. Apparatus according to claim 13, in which the dynamic range adjustment process is an inverse dynamic range adjustment process, in which the one or more processors are further configured to: decode the video data before performing the process of adjusting the inverse dynamic range on the luminance components and before performing the process of adjusting the inverse dynamic range on the chrominance components.
[26]
26. The apparatus according to claim 25, further comprising: a display configured to display the video data after performing the process of adjusting the dynamic dynamic range on the luminance components and after performing the process of adjusting the dynamic dynamic range on the components chrominance.
[27]
27. Apparatus configured to process video data, the apparatus comprising: means for receiving video data; means for determining luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data; means for performing a dynamic range adjustment process on the luminance components using the luma scale parameters; means for determining chroma scale parameters for chrominance components of the video data using a function of the luma scale parameters; and means for performing a dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters.
[28]
Apparatus according to claim 27, in which the luma scale parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, the apparatus further comprising: means for apply a linearization process to the discontinuous function to produce linearized luma scale parameters; and means for determining the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters.
[29]
29. Non-transitory, computer-readable storage medium that stores instructions that, when executed, cause one or more processors on a configured device to process video data to: receive video data; determining luma scale parameters for each of a plurality of code word value ranges for luminance components of the video data; perform a dynamic range adjustment process on the luminance components using the luma scale parameters; determine chroma scale parameters for chrominance components of video data using a function of the luma scale parameters; and perform a dynamic range adjustment process on the chrominance components of the video data using the chroma scale parameters.
[30]
30. Non-transitory computer-readable storage medium according to claim 29, wherein the luma scale parameters for each of the plurality of code word value ranges for the luminance components are represented by a discontinuous function, and where the instructions still make the one or more processors: apply a linearization process to the discontinuous function to produce linearized luma scale parameters; and determining the chroma scale parameters for the chrominance components of the video data using a function of the linearized luma scale parameters.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112020003037A2|2020-08-04|cross-component dynamic range adjustment system and method | in video encoding

---

RU2701961C2|2019-10-02|Dynamic range adjustment for video encoding with extended dynamic range and wide colour gamma

---

US11128878B2|2021-09-21|Fixed point implementation of range adjustment of components in video coding

---

KR20180056726A|2018-05-29|Supplemental Enhancement Information | messages for high dynamic range and wide gamut video coding

---

JP2018532316A|2018-11-01|Video decoder adaptation for high dynamic range | video coding using core video standards

---

KR20180016383A|2018-02-14|Content-adaptive application of fixed transfer function to high dynamic range | and/or wide color gamut | video data

---

BR112020006985A2|2020-10-06|video encoding with spatially variable quantization adaptable to content

---

US10681358B2|2020-06-09|Quantization parameter control for video coding with joined pixel/transform based quantization

---

BR112020013979A2|2020-12-08|SIGNALING MECHANISMS FOR EQUAL TRACKS AND OTHER DRA PARAMETERS FOR VIDEO ENCODING

---

KR20200140838A|2020-12-16|Harmony of transform-based quantization and dynamic range adjustment scale derivation in video coding

---

US10728559B2|2020-07-28|Precision of computation and signaling of dynamic range adjustment and color remapping information

---

TW201711470A|2017-03-16|Adaptive constant-luminance approach for high dynamic range and wide color gamut video coding

---

BR112020021315A2|2021-01-19|DECODED IMAGE BUFFER MANAGEMENT AND DYNAMIC RANGE ADJUSTMENT

---

US20180048907A1|2018-02-15|Video coding tools for in-loop sample processing

---

US20210329304A1|2021-10-21|Dynamic range adjustment parameter signaling and enablement of variable bit depth support

---

CA2979075A1|2016-10-20|Dynamic range adjustment for high dynamic range and wide color gamut video coding

同族专利:

---

公开号 | 公开日

---

US20190068969A1|2019-02-28|

---

EP3673652A1|2020-07-01|

---

AU2018322042A1|2020-02-06|

---

KR20200037272A|2020-04-08|

---

US10778978B2|2020-09-15|

---

TW201921928A|2019-06-01|

---

WO2019040502A1|2019-02-28|

---

SG11202000296TA|2020-03-30|

---

CN110999299A|2020-04-10|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US8457203B2|2005-05-26|2013-06-04|Ntt Docomo, Inc.|Method and apparatus for coding motion and prediction weighting parameters|

---

JP5066327B2|2005-06-28|2012-11-07|株式会社ジャパンディスプレイイースト|Liquid crystal display|

---

US20080247665A1|2007-04-04|2008-10-09|Silicon Integrated Systems Corp.|Method and apparatus for dynamic contrast enhancement|

---

WO2010105036A1|2009-03-13|2010-09-16|Dolby Laboratories Licensing Corporation|Layered compression of high dynamic range, visual dynamic range, and wide color gamut video|

---

US8831340B2|2010-01-27|2014-09-09|Adobe Systems Incorporated|Methods and apparatus for tone mapping high dynamic range images|

---

US8334911B2|2011-04-15|2012-12-18|Dolby Laboratories Licensing Corporation|Encoding, decoding, and representing high dynamic range images|

---

WO2013109471A1|2012-01-19|2013-07-25|Vid Scale, Inc.|System and method of video coding quantization and dynamic range control|

---

KR20130141920A|2012-06-18|2013-12-27|삼성디스플레이 주식회사|System and method for converting color gamut|

---

US9591302B2|2012-07-02|2017-03-07|Microsoft Technology Licensing, Llc|Use of chroma quantization parameter offsets in deblocking|

---

US9414054B2|2012-07-02|2016-08-09|Microsoft Technology Licensing, Llc|Control and use of chroma quantization parameter values|

---

US10218917B2|2013-07-16|2019-02-26|Koninklijke Philips N.V.|Method and apparatus to create an EOTF function for a universal code mapping for an HDR image, method and process to use these images|

---

JP6202330B2|2013-10-15|2017-09-27|ソニー株式会社|Decoding device and decoding method, and encoding device and encoding method|

---

KR101797505B1|2013-11-13|2017-12-12|엘지전자 주식회사|Broadcast signal transmission method and apparatus for providing hdr broadcast service|

---

JP2017512393A|2014-02-21|2017-05-18|コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ．|Video decoder enabling high resolution and high dynamic range|

---

US10419767B2|2014-02-21|2019-09-17|Koninklijke Philips N.V.|Encoding video with the luminances of the pixel colors converted into lumas with a predetermined code allocation and decoding the video|

---

EP3111644A1|2014-02-25|2017-01-04|Apple Inc.|Adaptive transfer function for video encoding and decoding|

---

JP2015170905A|2014-03-05|2015-09-28|ソニー株式会社|Image processor, image processing method and image display device|

---

WO2015180854A1|2014-05-28|2015-12-03|Koninklijke Philips N.V.|Methods and apparatuses for encoding an hdr images, and methods and apparatuses for use of such encoded images|

---

CN107111980B|2014-12-11|2021-03-09|皇家飞利浦有限公司|Optimizing high dynamic range images for specific displays|

---

US9860504B2|2015-01-09|2018-01-02|Vixs Systems, Inc.|Color gamut mapper for dynamic range conversion and methods for use therewith|

---

US9558538B2|2015-01-09|2017-01-31|Vixs Systems, Inc.|Dynamic range converter with frame by frame adaptation and methods for use therewith|

---

CN107409210B|2015-01-30|2021-01-12|交互数字Vc控股公司|Method and apparatus for matching colors between color pictures of different dynamic ranges|

---

EP3051818A1|2015-01-30|2016-08-03|Thomson Licensing|Method and device for decoding a color picture|

---

CN111654697A|2015-01-30|2020-09-11|交互数字Vc控股公司|Method and apparatus for encoding and decoding color picture|

---

WO2016154336A1|2015-03-25|2016-09-29|Dolby Laboratories Licensing Corporation|Chroma subsampling and gamut reshaping|

---

CN107534769B|2015-04-17|2021-08-03|交互数字麦迪逊专利控股公司|Chroma enhancement filtering for high dynamic range video coding|

---

US20180309995A1|2015-04-21|2018-10-25|Vid Scale, Inc.|High dynamic range video coding|

---

KR102059256B1|2015-06-05|2019-12-24|애플 인크.|Render and display HDR content|

---

WO2016199409A1|2015-06-07|2016-12-15|Sharp Kabushiki Kaisha|Systems and methods for optimizing video coding based on a luminance transfer function or video color component values|

---

EP3113495A1|2015-06-30|2017-01-04|Thomson Licensing|Methods and devices for encoding and decoding a hdr color picture|

---

WO2017040237A1|2015-08-28|2017-03-09|Arris Enterprises Llc|Color volume transforms in coding of high dynamic range and wide color gamut sequences|

---

WO2017037228A1|2015-09-02|2017-03-09|Thomson Licensing|Method and apparatus for quantization in video encoding and decoding|

---

US10244249B2|2015-09-21|2019-03-26|Qualcomm Incorporated|Fixed point implementation of range adjustment of components in video coding|

---

US10129558B2|2015-09-21|2018-11-13|Qualcomm Incorporated|Supplement enhancement information messages for high dynamic range and wide color gamut video coding|

---

EP3354019A4|2015-09-23|2019-05-22|Telefonaktiebolaget LM Ericsson |Determination of qp values|

---

US20170105014A1|2015-10-08|2017-04-13|Qualcomm Incorporated|Luma-driven chroma scaling for high dynamic range and wide color gamut contents|

---

US10043251B2|2015-10-09|2018-08-07|Stmicroelectronics Asia Pacific Pte Ltd|Enhanced tone mapper for high dynamic range images and video|

---

US10200701B2|2015-10-14|2019-02-05|Qualcomm Incorporated|HDR and WCG coding architecture with SDR backwards compatibility in a single bitstream for video coding|

---

EP3169071B1|2015-11-16|2020-01-29|InterDigital VC Holdings, Inc.|Backward-compatible encoding of a hdr picture|

---

US10863201B2|2015-12-21|2020-12-08|Koninklijke Philips N.V.|Optimizing high dynamic range images for particular displays|

---

EP3403406A1|2016-01-15|2018-11-21|VID SCALE, Inc.|System and method for enhanced motion compensation using adaptive filtering|

---

EP3409015B1|2016-01-28|2019-09-25|Koninklijke Philips N.V.|Encoding and decoding hdr videos|

---

JP6451669B2|2016-03-04|2019-01-16|ソニー株式会社|Evaluation apparatus, evaluation method, and camera system|

---

US10019785B2|2016-03-07|2018-07-10|Novatek Microelectronics Corp.|Method of processing high dynamic range images using dynamic metadata|

---

RU2728516C2|2016-03-18|2020-07-30|Конинклейке Филипс Н.В.|Hdr video encoding and decoding|

---

EP3446486B1|2016-04-22|2021-12-08|Dolby Laboratories Licensing Corporation|Coding of hdr video signals in the ictcp color format|

---

EP3453177A1|2016-05-04|2019-03-13|Interdigital VC Holdings, Inc|Method and apparatus for encoding/decoding a scalar integer into a parameter representative of a pivot points of a piece-wise linear function.|

---

JP6937781B2|2016-05-04|2021-09-22|インターデジタル ヴイシー ホールディングス， インコーポレイテッド|Methods and equipment for coding / decoding high dynamic range images into coded bitstreams|

---

EP3242482A1|2016-05-04|2017-11-08|Thomson Licensing|Method and apparatus for encoding/decoding a high dynamic range picture into a coded bitstream|

---

US10477212B2|2016-10-11|2019-11-12|Dolby Laboratories Licensing Corporation|Adaptive chroma quantization in video coding for multiple color imaging formats|

---

US10462334B2|2016-11-04|2019-10-29|Disney Enterprises, Inc.|Pipeline for high dynamic range video coding based on luminance independent chromaticity preprocessing|

---

US10148871B2|2016-12-07|2018-12-04|Microsoft Technology Licensing, Llc|Advanced raw conversion to produce high dynamic range, wide color gamut output|US10129558B2|2015-09-21|2018-11-13|Qualcomm Incorporated|Supplement enhancement informationmessages for high dynamic range and wide color gamut video coding|

---

US10244249B2|2015-09-21|2019-03-26|Qualcomm Incorporated|Fixed point implementation of range adjustment of components in video coding|

---

US10701375B2|2016-03-23|2020-06-30|Dolby Laboratories Licensing Corporation|Encoding and decoding reversible production-quality single-layer video signals|

---

US10728559B2|2017-07-07|2020-07-28|Qualcomm Incorporated|Precision of computation and signaling of dynamic range adjustment and color remapping information|

---

US10931977B2|2018-03-15|2021-02-23|Comcast Cable Communications, Llc|Systems, methods, and apparatuses for processing video|

---

US11153579B2|2018-03-23|2021-10-19|Qualcomm Incorporated|Deriving dynamic range adjustmentparameters for video coding|

---

KR20210134375A|2019-03-04|2021-11-09|알리바바 그룹 홀딩 리미티드|Methods and systems for processing video content|

---

AU2020237079A1|2019-03-12|2021-09-23|Tencent America LLC|Method and apparatus for video encoding or decoding|

---

US11122270B2|2019-06-05|2021-09-14|Dolby Laboratories Licensing Corporation|In-loop reshaping with local illumination compensation in image coding|

---

WO2020263493A1|2019-06-24|2020-12-30|Alibaba Group Holding Limited|Method and system for processing luma and chroma signals|

---

CN113396587A|2019-06-25|2021-09-14|Oppo广东移动通信有限公司|Image component prediction method, device and computer storage medium|

---

US20210281844A1|2020-03-05|2021-09-09|Qualcomm Incorporated|Methods for quantization parameter control for video coding with joined pixel/transform based quantization|

---

US20210329273A1|2020-04-18|2021-10-21|Qualcomm Incorporated|Flexible chroma processing for dynamic range adjustment|

---

CN111970564A|2020-08-26|2020-11-20|展讯通信（上海）有限公司|Optimization method and device for HDR video display processing, storage medium and terminal|

法律状态:
2021-11-03| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US201762548236P| true| 2017-08-21|2017-08-21|

---

US62/548,236|2017-08-21|

---

US15/999,393|US10778978B2|2017-08-21|2018-08-20|System and method of cross-component dynamic range adjustmentin video coding|

---

US15/999,393|2018-08-20|

---

PCT/US2018/047336|WO2019040502A1|2017-08-21|2018-08-21|System and method of cross-component dynamic range adjustmentin video coding|

---